Oct 29 23:10:32.895154 ip-10-0-138-189 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 29 23:10:32.895165 ip-10-0-138-189 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 29 23:10:32.895174 ip-10-0-138-189 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 29 23:10:32.895506 ip-10-0-138-189 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 29 23:10:43.274275 ip-10-0-138-189 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 29 23:10:43.274287 ip-10-0-138-189 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot bfc8ceae1f274652a4d4ffba5207a5c0 --
Oct 29 23:12:27.574024 ip-10-0-138-189 systemd[1]: Starting Kubernetes Kubelet...
Oct 29 23:12:27.964471 ip-10-0-138-189 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:12:27.964471 ip-10-0-138-189 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 29 23:12:27.964471 ip-10-0-138-189 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:12:27.964471 ip-10-0-138-189 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:12:27.969178 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969097    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.969178 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969177    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.969178 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969181    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969184    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969187    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969192    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969197    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969200    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969203    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969206    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969209    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969212    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969215    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969218    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969221    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969224    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969226    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969229    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969232    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969234    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969237    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.969274 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969240    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969243    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969245    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969248    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969251    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969254    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969256    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969259    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969263    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969265    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969268    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969270    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969273    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969276    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969279    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969282    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969285    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969288    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969291    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969293    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.969713 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969296    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969299    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969301    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969304    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969308    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969311    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969314    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969316    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969319    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969322    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969324    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969327    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969329    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969332    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969336    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969339    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969341    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969344    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969347    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969350    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.970197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969352    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969355    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969358    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969361    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969363    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969366    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969369    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969371    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969374    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969378    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969381    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969384    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969387    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969390    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969393    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969395    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969398    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969401    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969404    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.970673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969407    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969409    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969412    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969414    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969417    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969419    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969423    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969426    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969429    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969432    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969435    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969437    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969440    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969443    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969445    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969448    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969450    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969454    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969456    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969459    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.971168 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969462    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969465    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969467    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969470    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969474    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.969476    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970264    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970272    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970275    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970278    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970282    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970286    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970290    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970292    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970295    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970298    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970301    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970304    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970306    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.971635 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970311    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970315    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970318    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970321    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970324    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970327    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970330    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970333    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970335    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970338    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970341    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970343    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970346    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970349    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970351    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970354    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970356    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970359    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970361    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970364    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.972193 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970366    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970369    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970371    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970375    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970378    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970381    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970384    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970386    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970389    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970392    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970394    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970397    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970400    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970403    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970406    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970411    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970415    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970418    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970421    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.972673 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970423    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970426    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970429    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970432    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970434    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970437    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970439    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970442    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970445    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970447    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970450    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970452    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970455    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970458    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970460    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970463    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970466    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970469    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970471    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970474    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.973142 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970477    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970480    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970482    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970485    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970488    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970490    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970493    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970496    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970499    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970501    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970504    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970507    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970509    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970512    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970514    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970517    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970519    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970522    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970525    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970527    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.973604 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970531    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970534    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970536    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970539    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970542    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970544    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970547    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970549    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970552    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970554    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970557    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970559    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970571    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.970575    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971083    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971089    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971094    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971097    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971099    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.974092 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971102    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971105    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971108    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971110    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971113    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971115    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971118    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971120    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971123    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971126    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971128    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971131    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971134    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971137    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971140    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971143    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971146    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971148    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971151    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971154    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.974537 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971156    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971159    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971161    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971164    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971167    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971170    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971173    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971177    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971181    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971186    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971189    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971192    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971195    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971197    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971200    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971203    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971205    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971208    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971211    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.975042 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971213    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971218    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971222    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971225    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971228    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971231    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971234    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971237    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971239    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971241    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971244    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971247    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971250    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971252    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971255    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971257    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971260    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971262    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971265    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.975489 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971268    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971270    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971273    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971277    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971279    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971282    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971284    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971287    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971289    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971292    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971294    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971297    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971300    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971302    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971305    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971308    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971310    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971313    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971317    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971320    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.975972 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971322    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971325    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971327    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971330    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971333    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971335    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971338    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971340    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971343    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971346    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971348    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971351    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971353    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971357    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971359    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971363    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971365    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971368    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971371    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971373    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.976441 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971376    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971380    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.971382    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972522    2574 flags.go:64] FLAG: --address="0.0.0.0"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972533    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972539    2574 flags.go:64] FLAG: --anonymous-auth="true"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972545    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972549    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972552    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972557    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972564    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972567    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972570    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972574    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972577    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972581    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972584    2574 flags.go:64] FLAG: --cgroup-root=""
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972586    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972590    2574 flags.go:64] FLAG: --client-ca-file=""
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972593    2574 flags.go:64] FLAG: --cloud-provider="external"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972596    2574 flags.go:64] FLAG: --cluster-dns="[]"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972601    2574 flags.go:64] FLAG: --cluster-domain=""
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972605    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 29 23:12:27.976911 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972608    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972611    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972615    2574 flags.go:64] FLAG: --container-log-max-files="5"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972619    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972623    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972626    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972630    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972633    2574 flags.go:64] FLAG: --contention-profiling="false"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972636    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972639    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972643    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972646    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972650    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972653    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972656    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972659    2574 flags.go:64] FLAG: --enable-load-reader="false"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972662    2574 flags.go:64] FLAG: --enable-server="true"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972665    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972670    2574 flags.go:64] FLAG: --event-burst="100"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972674    2574 flags.go:64] FLAG: --event-qps="50"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972677    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972681    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972684    2574 flags.go:64] FLAG: --eviction-hard=""
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972688    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 29 23:12:27.977437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972691    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972694    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972697    2574 flags.go:64] FLAG: --eviction-soft=""
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972700    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972703    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972706    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972710    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972712    2574 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972715    2574 flags.go:64] FLAG: --fail-swap-on="true"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972718    2574 flags.go:64] FLAG: --feature-gates=""
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972722    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972725    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972731    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972744    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972747    2574 flags.go:64] FLAG: --healthz-port="10248"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972751    2574 flags.go:64] FLAG: --help="false"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972754    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-138-189.ec2.internal"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972758    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972761    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972764    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972767    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972771    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972774    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 29 23:12:27.978069 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972777    2574 flags.go:64] FLAG: --image-service-endpoint=""
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972780    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972783    2574 flags.go:64] FLAG: --kube-api-burst="100"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972786    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972790    2574 flags.go:64] FLAG: --kube-api-qps="50"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972793    2574 flags.go:64] FLAG: --kube-reserved=""
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972796    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972799    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972802    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972805    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972808    2574 flags.go:64] FLAG: --lock-file=""
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972811    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972814    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972817    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972823    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972826    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972829    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972832    2574 flags.go:64] FLAG: --logging-format="text"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972835    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972838    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972841    2574 flags.go:64] FLAG: --manifest-url=""
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972844    2574 flags.go:64] FLAG: --manifest-url-header=""
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972850    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972853    2574 flags.go:64] FLAG: --max-open-files="1000000"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972858    2574 flags.go:64] FLAG: --max-pods="110"
Oct 29 23:12:27.978623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972861    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972864    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972867    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972870    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972873    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972876    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972879    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972886    2574 flags.go:64] FLAG: --node-status-max-images="50"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972889    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972892    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972895    2574 flags.go:64] FLAG: --pod-cidr=""
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972899    2574 flags.go:64] FLAG: --pod-manifest-path=""
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972903    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972906    2574 flags.go:64] FLAG: --pods-per-core="0"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972909    2574 flags.go:64] FLAG: --port="10250"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972912    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972915    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07cee1894d1cf65f8"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972918    2574 flags.go:64] FLAG: --qos-reserved=""
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972921    2574 flags.go:64] FLAG: --read-only-port="10255"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972924    2574 flags.go:64] FLAG: --register-node="true"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972927    2574 flags.go:64] FLAG: --register-with-taints=""
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972931    2574 flags.go:64] FLAG: --registry-burst="10"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972934    2574 flags.go:64] FLAG: --registry-qps="5"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972936    2574 flags.go:64] FLAG: --reserved-cpus=""
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972939    2574 flags.go:64] FLAG: --reserved-memory=""
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972943    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 29 23:12:27.979277 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972946    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972950    2574 flags.go:64] FLAG: --rotate-certificates="false"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972953    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972956    2574 flags.go:64] FLAG: --runonce="false"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972959    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972962    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972965    2574 flags.go:64] FLAG: --seccomp-default="false"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972968    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972971    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972974    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972977    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972980    2574 flags.go:64] FLAG: --storage-driver-password="root"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972983    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972986    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972989    2574 flags.go:64] FLAG: --storage-driver-user="root"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972992    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972995    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.972999    2574 flags.go:64] FLAG: --system-cgroups=""
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973002    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973007    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973011    2574 flags.go:64] FLAG: --tls-cert-file=""
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973013    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973017    2574 flags.go:64] FLAG: --tls-min-version=""
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973020    2574 flags.go:64] FLAG: --tls-private-key-file=""
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973023    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 29 23:12:27.979949 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973026    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973029    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973032    2574 flags.go:64] FLAG: --v="2"
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973036    2574 flags.go:64] FLAG: --version="false"
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973040    2574 flags.go:64] FLAG: --vmodule=""
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973044    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973048    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973209    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973215    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973218    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973221    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973223    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973227    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973229    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973232    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973235    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973238    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973240    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973243    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973245    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973248    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.980523 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973251    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973253    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973256    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973259    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973262    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973265    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973268    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973271    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973273    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973276    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973279    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973281    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973284    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973286    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973289    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973292    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973294    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973297    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973299    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973303    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.981031 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973306    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973308    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973311    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973314    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973317    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973319    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973322    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973325    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973327    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973330    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973333    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973336    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973338    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973341    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973344    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973346    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973349    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973352    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973354    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973357    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.981502 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973359    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973362    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973364    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973367    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973370    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973372    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973375    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973377    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973380    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973383    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973385    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973389    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973392    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973394    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973397    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973399    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973402    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973405    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973408    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.982021 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973410    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973413    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973415    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973418    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973421    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973424    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973426    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973429    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973431    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973434    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973437    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973439    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973442    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973455    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973459    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973462    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973465    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973468    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973470    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.982463 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973473    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973476    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973479    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973483    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973487    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973493    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973496    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973499    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973502    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973505    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973507    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973510    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973513    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973516    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.973543    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:12:27.982928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973579    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973583    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973586    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973589    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973592    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973595    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973598    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973600    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973603    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973605    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973608    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973611    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973614    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973617    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973620    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973623    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973626    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973629    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973631    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973634    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.983281 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973636    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973639    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973643    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973648    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973652    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973656    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973659    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973662    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973665    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973669    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973673    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973676    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973678    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973681    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973683    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973686    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973688    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973691    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973694    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.983766 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973696    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973699    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973701    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973704    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973706    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973709    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973711    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973714    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973717    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973719    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973722    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973724    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973727    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973729    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973732    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973747    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973751    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973754    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973756    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973759    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.984214 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973762    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973764    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973769    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973771    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973775    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973777    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973780    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973782    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973785    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973788    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973790    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973793    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973795    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973798    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973801    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973803    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973806    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973808    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973813    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973815    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.984722 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973818    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973832    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973836    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973839    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973842    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973846    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973848    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973851    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973855    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973858    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973860    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973863    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973865    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973868    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973872    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973875    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973877    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973881    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973883    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973887    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.985322 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973890    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973892    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973895    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973897    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973900    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973902    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.973905    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.981240    2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.981256    2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.981995    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.982003    2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982056    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982062    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982065    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982069    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982072    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982075    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982078    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982081    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.986122 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982084    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982087    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982089    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982092    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982095    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982098    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982100    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982103    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982106    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982109    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982113    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982116    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982119    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982122    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982125    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982128    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982130    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982133    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982136    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982139    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.986874 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982141    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982145    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982149    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982151    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982154    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982157    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982160    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982162    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982165    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982167    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982170    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982173    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982175    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982178    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982180    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982183    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982186    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982188    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982191    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982193    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.987508 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982196    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982199    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982201    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982204    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982206    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982209    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982211    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982214    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982217    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982219    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982222    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982225    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982227    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982230    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982234    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982236    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982239    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982242    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982245    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.988055 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982247    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982250    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982253    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982255    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982258    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982262    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982265    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982269    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982272    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982275    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982277    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982280    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982283    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982285    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982288    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982290    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982293    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982296    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982298    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.988547 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982301    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982304    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982306    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982309    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982312    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982314    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982316    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982320    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982322    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982325    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982328    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982331    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982333    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982336    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982339    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982341    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982344    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982346    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982349    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.989013 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982351    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.982379    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982506    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982510    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982513    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982516    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982519    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982522    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982524    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982527    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982530    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982532    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982535    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982538    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982540    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.989512 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982543    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982546    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982549    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982551    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982554    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982556    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982559    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982562    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982574    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982577    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982579    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982582    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982585    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982588    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982590    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982593    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982596    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982599    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982601    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982604    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.989942 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982606    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982609    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982612    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982616    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982620    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982623    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982625    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982628    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982630    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982633    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982636    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982638    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982641    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982644    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982646    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982649    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982652    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982655    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982657    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982660    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.990583 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982662    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982666    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982670    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982673    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982675    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982678    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982681    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982684    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982687    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982689    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982692    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982694    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982697    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982700    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982702    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982705    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982707    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982709    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982712    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.991399 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982715    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982717    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982720    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982722    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982726    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982728    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982731    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982733    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982751    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982754    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982756    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982759    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982762    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982764    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982767    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982770    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982773    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982776    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982778    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.992197 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982780    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982783    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982786    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982788    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982791    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982793    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982796    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982798    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982801    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982803    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982806    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982808    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982811    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982813    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982816    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.982840    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:12:27.992904 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982869    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982874    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982876    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982879    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982882    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982884    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982887    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982890    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982893    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982895    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982898    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982900    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982903    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982906    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982909    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982912    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982914    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982917    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982919    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982922    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.993296 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982924    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982927    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982930    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982932    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982935    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982938    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982941    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982943    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982946    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982948    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982951    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982954    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982956    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982959    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982961    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982964    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982967    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982969    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982972    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.993798 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982974    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982977    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982979    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982982    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982984    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982987    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982989    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982992    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982995    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.982997    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983000    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983002    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983005    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983008    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983010    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983013    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983015    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983018    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983020    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983023    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.994244 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983026    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983028    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983031    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983035    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983039    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983041    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983044    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983047    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983049    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983052    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983055    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983057    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983060    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983062    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983065    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983067    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983070    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983072    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983075    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983077    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.994861 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983081    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983085    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983089    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983092    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983094    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983097    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983100    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983102    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983105    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983107    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983110    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983112    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983115    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983117    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983119    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983122    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983125    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983127    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983130    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.995315 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983132    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983135    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983138    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983140    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983143    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983146    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983148    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:27.983151    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.983201    2574 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.987097    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.988356    2574 server.go:1014] "Starting client certificate rotation"
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.988457    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.988494    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.990838    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 29 23:12:27.995797 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:27.992686    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 29 23:12:28.005918 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.005898    2574 log.go:25] "Validated CRI v1 runtime API"
Oct 29 23:12:28.009873 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.009856    2574 log.go:25] "Validated CRI v1 image API"
Oct 29 23:12:28.010750 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.010715    2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 29 23:12:28.017623 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.017601    2574 fs.go:135] Filesystem UUIDs: map[153ded04-b3e4-47c1-ac22-6c3834ae04da:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b0856242-c573-4082-86de-174e5385a837:/dev/nvme0n1p4]
Oct 29 23:12:28.017711 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.017622    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 29 23:12:28.023950 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.023825    2574 manager.go:217] Machine: {Timestamp:2026-10-29 23:12:28.021489657 +0000 UTC m=+0.360947327 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102722 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec239fb954fbcd98fb75b56338fa5772 SystemUUID:ec239fb9-54fb-cd98-fb75-b56338fa5772 BootID:bfc8ceae-1f27-4652-a4d4-ffba5207a5c0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f4:56:1c:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f4:56:1c:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:ba:59:b7:bf:5f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 29 23:12:28.024604 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.024591    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 29 23:12:28.024728 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.024705    2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 29 23:12:28.028649 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.028620    2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 29 23:12:28.028824 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.028650    2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-189.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 29 23:12:28.028919 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.028835    2574 topology_manager.go:143] "Creating topology manager with none policy"
Oct 29 23:12:28.028919 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.028851    2574 container_manager_linux.go:311] "Creating device plugin manager"
Oct 29 23:12:28.028919 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.028887    2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 29 23:12:28.030692 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.030679    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 29 23:12:28.031903 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.031889    2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 29 23:12:28.036748 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.036720    2574 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 29 23:12:28.036885 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.036875    2574 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 29 23:12:28.037785 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.037776    2574 kubelet.go:493] "Attempting to sync node with API server"
Oct 29 23:12:28.037826 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.037793    2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 29 23:12:28.037826 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.037803    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 29 23:12:28.037826 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.037812    2574 kubelet.go:405] "Adding apiserver pod source"
Oct 29 23:12:28.037826 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.037820    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 29 23:12:28.038383 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.038369    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 29 23:12:28.042372 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.042163    2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 29 23:12:28.049187 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.049175    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 29 23:12:28.050025 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050013    2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 29 23:12:28.050062 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050043    2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 29 23:12:28.050190 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050181    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 29 23:12:28.050218 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050195    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 29 23:12:28.050218 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050202    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 29 23:12:28.050218 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050207    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 29 23:12:28.050292 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050220    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 29 23:12:28.050292 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050227    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 29 23:12:28.050887 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050874    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 29 23:12:28.050924 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050890    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 29 23:12:28.050924 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050902    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 29 23:12:28.050924 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050911    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 29 23:12:28.050924 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050923    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 29 23:12:28.051077 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.050933    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 29 23:12:28.051806 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.051792    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 29 23:12:28.051849 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.051811    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 29 23:12:28.055621 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.055607    2574 server.go:1263] "Started kubelet"
Oct 29 23:12:28.055809 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.055755    2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 29 23:12:28.055809 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.055727    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 29 23:12:28.055943 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.055865    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 29 23:12:28.056520 ip-10-0-138-189 systemd[1]: Started Kubernetes Kubelet.
Oct 29 23:12:28.057639 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.057519    2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 29 23:12:28.058193 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.058179    2574 server.go:317] "Adding debug handlers to kubelet server"
Oct 29 23:12:28.062377 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.062359    2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 29 23:12:28.069514 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.069491    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 29 23:12:28.070195 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.070179    2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 29 23:12:28.070322 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.070296    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:12:28.071220 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071205    2574 factory.go:55] Registering systemd factory
Oct 29 23:12:28.071220 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071220    2574 factory.go:223] Registration of the systemd container factory successfully
Oct 29 23:12:28.071389 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071362    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 29 23:12:28.071389 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071359    2574 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 29 23:12:28.071484 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071403    2574 factory.go:153] Registering CRI-O factory
Oct 29 23:12:28.071484 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071405    2574 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 29 23:12:28.071484 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071418    2574 factory.go:223] Registration of the crio container factory successfully
Oct 29 23:12:28.071594 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071522    2574 reconstruct.go:97] "Volume reconstruction finished"
Oct 29 23:12:28.071594 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071529    2574 reconciler.go:29] "Reconciler: start to sync state"
Oct 29 23:12:28.071594 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071529    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 29 23:12:28.071594 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071553    2574 factory.go:103] Registering Raw factory
Oct 29 23:12:28.071594 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.071565    2574 manager.go:1199] Started watching for new ooms in manager
Oct 29 23:12:28.072630 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.072614    2574 manager.go:319] Starting recovery of all containers
Oct 29 23:12:28.081025 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.080991    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 29 23:12:28.081181 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.081163    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8l6k8"
Oct 29 23:12:28.084764 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.084730    2574 manager.go:324] Recovery completed
Oct 29 23:12:28.086432 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.086401    2574 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-189.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 29 23:12:28.086432 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.086400    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-189.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 29 23:12:28.087569 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.086397    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-189.ec2.internal.18e322fd5409b2a3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-189.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-189.ec2.internal,},FirstTimestamp:2026-10-29 23:12:28.055581347 +0000 UTC m=+0.395039018,LastTimestamp:2026-10-29 23:12:28.055581347 +0000 UTC m=+0.395039018,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-189.ec2.internal,}"
Oct 29 23:12:28.090024 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.090009    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:28.091647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.091630    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:28.091703 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.091662    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:28.091703 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.091676    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:28.092220 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.092200    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8l6k8"
Oct 29 23:12:28.092307 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.092251    2574 cpu_manager.go:226] "Starting" policy="none"
Oct 29 23:12:28.092307 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.092260    2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 29 23:12:28.092307 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.092277    2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 29 23:12:28.092307 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.092300    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:28.093054 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.093039    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:28.093132 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.093067    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:28.093132 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.093081    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:28.095754 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.095727    2574 policy_none.go:50] "Start"
Oct 29 23:12:28.095754 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.095754    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 29 23:12:28.095876 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.095764    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 29 23:12:28.098238 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.098226    2574 policy_none.go:44] "Start"
Oct 29 23:12:28.104299 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.104272    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 29 23:12:28.104380 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.104271    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-189.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.136355    2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.139168    2574 manager.go:341] "Starting Device Plugin manager"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.139194    2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.139202    2574 server.go:87] "Starting device plugin registration server"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.139479    2574 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.139490    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.139562    2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.139626    2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.139634    2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.140348    2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.140390    2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.144047    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.145284    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.145304    2574 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.145326    2574 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 29 23:12:28.148022 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.145364    2574 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 29 23:12:28.153093 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.153076    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:12:28.161944 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.161928    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:12:28.240155 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.239968    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:28.241104 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.241087    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:28.241190 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.241115    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:28.241190 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.241130    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:28.241190 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.241170    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.246482 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.246465    2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"]
Oct 29 23:12:28.246531 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.246519    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:28.248933 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.248918    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:28.249000 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.248945    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:28.249000 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.248957    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:28.251152 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.251141    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:28.251746 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.251719    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.251828 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.251770    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:28.251828 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.251796    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:28.251828 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.251818    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:28.251828 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.251828    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:28.252437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.252424    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:28.252493 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.252450    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:28.252493 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.252461    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:28.253895 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.253879    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.253988 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.253903    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:28.254505 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.254489    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:28.254590 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.254516    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:28.254590 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.254528    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:28.259994 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.259978    2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.260072 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.259999    2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-189.ec2.internal\": node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:12:28.272625 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.272606    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.272692 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.272631    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.272692 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.272649    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2f6cf61a946e2452e67c50b8a0479c0d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-189.ec2.internal\" (UID: \"2f6cf61a946e2452e67c50b8a0479c0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.282389 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.282374    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-189.ec2.internal\" not found" node="ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.285197 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.285183    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-189.ec2.internal\" not found" node="ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.312838 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.312818    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:12:28.373605 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.373580    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.373704 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.373621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.373704 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.373667    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.373704 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.373684    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2f6cf61a946e2452e67c50b8a0479c0d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-189.ec2.internal\" (UID: \"2f6cf61a946e2452e67c50b8a0479c0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.373818 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.373721    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2f6cf61a946e2452e67c50b8a0479c0d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-189.ec2.internal\" (UID: \"2f6cf61a946e2452e67c50b8a0479c0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.373818 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.373771    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.413850 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.413829    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:12:28.514255 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.514190    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:12:28.587643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.587619    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.590087 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.590072    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:28.614463 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.614441    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:12:28.716891 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.716836    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:12:28.817531 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.817449    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:12:28.919957 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:28.919918    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:12:28.988321 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.988284    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 29 23:12:28.988865 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.988458    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 29 23:12:28.988865 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:28.988490    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 29 23:12:29.020960 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:29.020924    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:12:29.098392 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:29.098288    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-28 23:07:28 +0000 UTC" deadline="2028-08-11 22:52:45.273765932 +0000 UTC"
Oct 29 23:12:29.098392 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:29.098333    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15647h40m16.175436079s"
Oct 29 23:12:29.111329 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:29.111307    2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:12:29.171017 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:29.170966    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:29.198278 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:29.198248    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:12:29.198410 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:29.198372    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:12:29.222663 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:29.222640    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:12:29.247948 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:29.247908    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa79b04c65fcc2c4af3f0c1b29fc1ed6.slice/crio-5bdb8aa5c502505aee9639e14dbb739a31238777723dbee7f9e274f97570f86c WatchSource:0}: Error finding container 5bdb8aa5c502505aee9639e14dbb739a31238777723dbee7f9e274f97570f86c: Status 404 returned error can't find the container with id 5bdb8aa5c502505aee9639e14dbb739a31238777723dbee7f9e274f97570f86c
Oct 29 23:12:29.248382 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:29.248357    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6cf61a946e2452e67c50b8a0479c0d.slice/crio-003e85709816678966b319fe60adb5ce0a64a71563dcc1ee5febfd0126b2130b WatchSource:0}: Error finding container 003e85709816678966b319fe60adb5ce0a64a71563dcc1ee5febfd0126b2130b: Status 404 returned error can't find the container with id 003e85709816678966b319fe60adb5ce0a64a71563dcc1ee5febfd0126b2130b
Oct 29 23:12:29.253738 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:29.253721    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 29 23:12:29.488847 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:29.488740    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:12:30.039269 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.039241    2574 apiserver.go:52] "Watching apiserver"
Oct 29 23:12:30.070140 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.070111    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 29 23:12:30.072893 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.072867    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:12:30.087362 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.087338    2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 29 23:12:30.087707 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.087683    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-5z4zk","openshift-cluster-node-tuning-operator/tuned-nsxh7","openshift-image-registry/node-ca-tn9mk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal","openshift-multus/multus-95nsq","openshift-network-operator/iptables-alerter-vk7hb","kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-fk4wd","kube-system/global-pull-secret-syncer-wht7s","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w","openshift-dns/node-resolver-n4lkv","openshift-multus/multus-additional-cni-plugins-wmfk5","openshift-multus/network-metrics-daemon-bv5tv","openshift-network-diagnostics/network-check-target-xhlf6"]
Oct 29 23:12:30.090441 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.090416    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:30.090548 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.090512    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:30.092928 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.092905    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.095271 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.095249    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:12:30.098059 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.098042    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:30.100151 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.100131    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.102718 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.102499    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:12:30.107274 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.106913    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:30.107274 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.106987    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:30.108929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.108906    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.109556 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.109529    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:12:30.109657 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.109584    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 29 23:12:30.109657 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.109627    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kcr8p\""
Oct 29 23:12:30.111230 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.111210    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 29 23:12:30.111332 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.111316    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w5p5k\""
Oct 29 23:12:30.111428 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.111385    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 29 23:12:30.111487 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.111432    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 29 23:12:30.111992 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.111970    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:30.112080 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.112040    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:30.114735 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.114723    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.116748 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.116720    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:12:30.119597 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.119325    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.122840 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.120580    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 29 23:12:30.122840 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.120932    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 29 23:12:30.122840 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.121032    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 29 23:12:30.122840 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.121352    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-5wvkd\""
Oct 29 23:12:30.122840 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.121543    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 29 23:12:30.122840 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.121615    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 29 23:12:30.122840 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.122128    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zfk9z\""
Oct 29 23:12:30.122840 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.122485    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sw5tb\""
Oct 29 23:12:30.122840 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.122607    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 29 23:12:30.122840 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.122728    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:12:30.123457 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.123001    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 29 23:12:30.123457 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.123298    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 29 23:12:30.123457 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.123447    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tkkkp"
Oct 29 23:12:30.123592 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.123527    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 29 23:12:30.134976 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.134953    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tkkkp"
Oct 29 23:12:30.153221 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.153168    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal" event={"ID":"2f6cf61a946e2452e67c50b8a0479c0d","Type":"ContainerStarted","Data":"003e85709816678966b319fe60adb5ce0a64a71563dcc1ee5febfd0126b2130b"}
Oct 29 23:12:30.155727 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.155702    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal" event={"ID":"aa79b04c65fcc2c4af3f0c1b29fc1ed6","Type":"ContainerStarted","Data":"5bdb8aa5c502505aee9639e14dbb739a31238777723dbee7f9e274f97570f86c"}
Oct 29 23:12:30.172958 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.172934    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 29 23:12:30.185543 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185520    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-tuned\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.185640 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185549    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-system-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.185640 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185565    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-dbus\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:30.185640 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185587    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-env-overrides\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.185793 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185642    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.185793 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185666    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-conf\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.185793 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185687    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-lib-modules\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.185793 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185742    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-tmp\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.186003 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185802    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lb7c\" (UniqueName: \"kubernetes.io/projected/916781d5-af66-4d17-ba3f-748bab236eb1-kube-api-access-9lb7c\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:12:30.186003 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185855    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-cni-binary-copy\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.186003 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185884    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-kubelet\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.186003 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185910    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:30.186003 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.185964    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-kubernetes\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.186255 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186025    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/916781d5-af66-4d17-ba3f-748bab236eb1-serviceca\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:12:30.186255 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186063    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-bin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.186255 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186112    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-multus\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.186255 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186138    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-multus-certs\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.186255 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186165    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-ovn\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.186255 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeeba32d-22c4-4173-83f0-5b4c391644b0-tmp-dir\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:12:30.186553 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186254    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.186553 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186286    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.186553 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186321    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.186553 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186348    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-etc-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.186553 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186376    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffjqr\" (UniqueName: \"kubernetes.io/projected/87b4272a-c0a8-4ff8-8b80-cf9d76552032-kube-api-access-ffjqr\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.186553 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186401    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-modprobe-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.186553 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186430    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-sys\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.186553 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186465    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-host\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.186553 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186482    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-etc-kubernetes\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.186553 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186502    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk7h7\" (UniqueName: \"kubernetes.io/projected/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-kube-api-access-qk7h7\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:12:30.186553 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186528    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-registration-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.187068 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186588    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-etc-selinux\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.187068 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-sys-fs\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.187068 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186661    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysconfig\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.187068 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186696    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-konnectivity-ca\") pod \"konnectivity-agent-5z4zk\" (UID: \"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e\") " pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:30.187068 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186801    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-host-slash\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:12:30.187068 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186855    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-netd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.187068 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186885    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2wfh\" (UniqueName: \"kubernetes.io/projected/54081aac-51a7-47aa-8dab-a04c8d68ab5f-kube-api-access-b2wfh\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.187068 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/916781d5-af66-4d17-ba3f-748bab236eb1-host\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:12:30.187068 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.186977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-systemd-units\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.187068 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187014    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-script-lib\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.187068 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187068    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-conf-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187106    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpjpd\" (UniqueName: \"kubernetes.io/projected/2bc4c683-845b-40f3-ac2e-2c785b221257-kube-api-access-vpjpd\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187146    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-node-log\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187181    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-bin\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187212    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpcwp\" (UniqueName: \"kubernetes.io/projected/eeeba32d-22c4-4173-83f0-5b4c391644b0-kube-api-access-jpcwp\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187239    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-daemon-config\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187291    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-systemd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187338    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-config\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187364    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-log-socket\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187390    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-netns\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187415    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187451    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-cnibin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187484    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-netns\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187510    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-kubelet-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.187589 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187547    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187620    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187646    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-var-lib-kubelet\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187680    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-agent-certs\") pod \"konnectivity-agent-5z4zk\" (UID: \"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e\") " pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187712    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-socket-dir-parent\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187737    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-k8s-cni-cncf-io\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187761    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-slash\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187818    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxlfh\" (UniqueName: \"kubernetes.io/projected/22013da1-6ec7-4b6d-95f8-298e97fb1372-kube-api-access-mxlfh\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187842    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-hostroot\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187870    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187896    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovn-node-metrics-cert\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187925    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-kubelet-config\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.187951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cnibin\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188003    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-socket-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188030    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-device-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188136    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eeeba32d-22c4-4173-83f0-5b4c391644b0-hosts-file\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:12:30.188290 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188163    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-systemd\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.189131 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188189    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-kubelet\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.189131 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188216    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-iptables-alerter-script\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:12:30.189131 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-var-lib-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.189131 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188270    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:30.189131 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188296    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfnrq\" (UniqueName: \"kubernetes.io/projected/3f949f77-5aa6-4443-a09b-6fed361a0d43-kube-api-access-rfnrq\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.189131 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188322    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-system-cni-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.189131 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188359    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.189131 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188387    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcxgr\" (UniqueName: \"kubernetes.io/projected/62ec6811-5291-4963-b8f1-52a7ffdc50bc-kube-api-access-gcxgr\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:30.189131 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.188411    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-run\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.218632 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218608    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 29 23:12:30.218818 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218711    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 29 23:12:30.218818 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218781    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 29 23:12:30.218956 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218848    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 29 23:12:30.218956 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218849    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vsvpd\""
Oct 29 23:12:30.218956 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218886    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-96xtz\""
Oct 29 23:12:30.218956 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218903    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 29 23:12:30.218956 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218926    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7fjkb\""
Oct 29 23:12:30.218956 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218935    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 29 23:12:30.218956 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218960    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 29 23:12:30.219281 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218982    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 29 23:12:30.219281 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218985    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 29 23:12:30.219281 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218804    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 29 23:12:30.219281 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.219017    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 29 23:12:30.219281 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.218803    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 29 23:12:30.219281 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.219063    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 29 23:12:30.219281 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.219080    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h2str\""
Oct 29 23:12:30.289094 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289058    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-systemd\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.289094 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289100    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-kubelet\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.289370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289127    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-iptables-alerter-script\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:12:30.289370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289151    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-var-lib-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.289370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289205    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-var-lib-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.289370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289205    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-systemd\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.289370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289213    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-kubelet\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.289370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:30.289370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289345    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfnrq\" (UniqueName: \"kubernetes.io/projected/3f949f77-5aa6-4443-a09b-6fed361a0d43-kube-api-access-rfnrq\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.289370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289364    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-system-cni-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289385    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289403    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcxgr\" (UniqueName: \"kubernetes.io/projected/62ec6811-5291-4963-b8f1-52a7ffdc50bc-kube-api-access-gcxgr\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289417    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-run\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289432    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-tuned\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289452    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-system-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289471    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-dbus\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289488    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-env-overrides\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289504    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289525    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-conf\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289547    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-lib-modules\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-tmp\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.289655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lb7c\" (UniqueName: \"kubernetes.io/projected/916781d5-af66-4d17-ba3f-748bab236eb1-kube-api-access-9lb7c\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289662    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-cni-binary-copy\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289678    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-kubelet\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289696    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-kubernetes\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289729    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/916781d5-af66-4d17-ba3f-748bab236eb1-serviceca\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289751    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-bin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-multus\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289797    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-multus-certs\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-ovn\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289829    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeeba32d-22c4-4173-83f0-5b4c391644b0-tmp-dir\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289846    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289870    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289899    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-etc-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289958    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffjqr\" (UniqueName: \"kubernetes.io/projected/87b4272a-c0a8-4ff8-8b80-cf9d76552032-kube-api-access-ffjqr\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289964    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-lib-modules\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.290075 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-modprobe-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290024    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-sys\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290055    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-host\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290070    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-modprobe-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290083    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-etc-kubernetes\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290114    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk7h7\" (UniqueName: \"kubernetes.io/projected/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-kube-api-access-qk7h7\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290142    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-registration-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290184    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-etc-selinux\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290212    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-sys-fs\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290238    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysconfig\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290264    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-konnectivity-ca\") pod \"konnectivity-agent-5z4zk\" (UID: \"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e\") " pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290291    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-host-slash\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290317    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-netd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290338    2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290347    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2wfh\" (UniqueName: \"kubernetes.io/projected/54081aac-51a7-47aa-8dab-a04c8d68ab5f-kube-api-access-b2wfh\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290366    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290374    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/916781d5-af66-4d17-ba3f-748bab236eb1-host\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290418    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/916781d5-af66-4d17-ba3f-748bab236eb1-host\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:12:30.290807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290423    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-systemd-units\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290455    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-script-lib\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290549    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-conf\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290601    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-sys\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290697    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-system-cni-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.290716    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290774    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-etc-kubernetes\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290820    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290947    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-run\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290964    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-host\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290968    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-conf-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.290996    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysconfig\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291041    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-multus-certs\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.291031    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret podName:a8ac2ba2-0d4b-4498-a7c6-658412d60ccf nodeName:}" failed. No retries permitted until 2026-10-29 23:12:30.790984694 +0000 UTC m=+3.130442370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret") pod "global-pull-secret-syncer-wht7s" (UID: "a8ac2ba2-0d4b-4498-a7c6-658412d60ccf") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291081    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-registration-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291080    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-host-slash\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291150    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-sys-fs\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.291723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291228    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-etc-selinux\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291275    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-kubelet\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291474    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-kubernetes\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291527    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-konnectivity-ca\") pod \"konnectivity-agent-5z4zk\" (UID: \"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e\") " pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291542    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-dbus\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291594    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-env-overrides\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291615    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-system-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291647    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-systemd-units\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291656    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-conf-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291680    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-netd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291707    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-bin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291726    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpjpd\" (UniqueName: \"kubernetes.io/projected/2bc4c683-845b-40f3-ac2e-2c785b221257-kube-api-access-vpjpd\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291779    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-node-log\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291835    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-bin\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291867    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpcwp\" (UniqueName: \"kubernetes.io/projected/eeeba32d-22c4-4173-83f0-5b4c391644b0-kube-api-access-jpcwp\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291891    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-cni-binary-copy\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291891    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291916    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-node-log\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.292643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291928    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/916781d5-af66-4d17-ba3f-748bab236eb1-serviceca\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291946    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.291948    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-daemon-config\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292008    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-systemd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292014    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-multus\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292029    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-bin\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292041    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-config\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292068    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-log-socket\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292071    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-ovn\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292092    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.289809    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-iptables-alerter-script\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292166    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-systemd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292355    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-etc-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292591    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-netns\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-cnibin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292669    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-netns\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.293519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292695    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-kubelet-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292756    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292782    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-var-lib-kubelet\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292808    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-agent-certs\") pod \"konnectivity-agent-5z4zk\" (UID: \"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e\") " pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292851    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-socket-dir-parent\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292879    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-k8s-cni-cncf-io\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292905    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-slash\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292930    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxlfh\" (UniqueName: \"kubernetes.io/projected/22013da1-6ec7-4b6d-95f8-298e97fb1372-kube-api-access-mxlfh\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292962    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-hostroot\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.292989    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.293017    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovn-node-metrics-cert\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.293045    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-kubelet-config\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.293073    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cnibin\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.293100    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-socket-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.293125    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-device-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.293151    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eeeba32d-22c4-4173-83f0-5b4c391644b0-hosts-file\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:12:30.294294 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.293615    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.293641    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.293702    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeeba32d-22c4-4173-83f0-5b4c391644b0-tmp-dir\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.293723    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs podName:62ec6811-5291-4963-b8f1-52a7ffdc50bc nodeName:}" failed. No retries permitted until 2026-10-29 23:12:30.79370528 +0000 UTC m=+3.133162948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs") pod "network-metrics-daemon-bv5tv" (UID: "62ec6811-5291-4963-b8f1-52a7ffdc50bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.293755    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-daemon-config\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.293805    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-var-lib-kubelet\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294092    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294189    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294537    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294620    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-slash\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294637    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-tmp\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294685    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-socket-dir-parent\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294721    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-k8s-cni-cncf-io\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294746    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-netns\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294749    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-script-lib\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294801    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-log-socket\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294836    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-cnibin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.295163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294862    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-kubelet-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.295929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294883    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-config\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.295929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294901    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.295929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294951    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-netns\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.295929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294956    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-hostroot\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.295929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.294987    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-socket-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.295929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.295028    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-kubelet-config\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:30.295929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.295035    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cnibin\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.295929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.295031    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eeeba32d-22c4-4173-83f0-5b4c391644b0-hosts-file\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:12:30.295929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.295076    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-device-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.297106 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.297076    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovn-node-metrics-cert\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.299907 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.299884    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-agent-certs\") pod \"konnectivity-agent-5z4zk\" (UID: \"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e\") " pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:30.300417 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.300399    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-tuned\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.317486 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.317456    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:30.317486 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.317488    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:30.317685 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.317505    2574 projected.go:196] Error preparing data for projected volume kube-api-access-jbssw for pod openshift-network-diagnostics/network-check-target-xhlf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:30.317685 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.317574    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw podName:fd1c395e-bc72-4038-80cc-4ab131d96fdb nodeName:}" failed. No retries permitted until 2026-10-29 23:12:30.817553819 +0000 UTC m=+3.157011497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jbssw" (UniqueName: "kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw") pod "network-check-target-xhlf6" (UID: "fd1c395e-bc72-4038-80cc-4ab131d96fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:30.320604 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.320578    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk7h7\" (UniqueName: \"kubernetes.io/projected/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-kube-api-access-qk7h7\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:12:30.321409 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.321364    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpcwp\" (UniqueName: \"kubernetes.io/projected/eeeba32d-22c4-4173-83f0-5b4c391644b0-kube-api-access-jpcwp\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:12:30.321485 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.321427    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2wfh\" (UniqueName: \"kubernetes.io/projected/54081aac-51a7-47aa-8dab-a04c8d68ab5f-kube-api-access-b2wfh\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.322059 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.322035    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcxgr\" (UniqueName: \"kubernetes.io/projected/62ec6811-5291-4963-b8f1-52a7ffdc50bc-kube-api-access-gcxgr\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:30.322164 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.322042    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lb7c\" (UniqueName: \"kubernetes.io/projected/916781d5-af66-4d17-ba3f-748bab236eb1-kube-api-access-9lb7c\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:12:30.322858 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.322832    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpjpd\" (UniqueName: \"kubernetes.io/projected/2bc4c683-845b-40f3-ac2e-2c785b221257-kube-api-access-vpjpd\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.323421 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.323369    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfnrq\" (UniqueName: \"kubernetes.io/projected/3f949f77-5aa6-4443-a09b-6fed361a0d43-kube-api-access-rfnrq\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.324781 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.324756    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffjqr\" (UniqueName: \"kubernetes.io/projected/87b4272a-c0a8-4ff8-8b80-cf9d76552032-kube-api-access-ffjqr\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.325109 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.325086    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxlfh\" (UniqueName: \"kubernetes.io/projected/22013da1-6ec7-4b6d-95f8-298e97fb1372-kube-api-access-mxlfh\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.407581 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.407547    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:12:30.419289 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:30.419233    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22013da1_6ec7_4b6d_95f8_298e97fb1372.slice/crio-f36e3fe7651e3b7423e250a30bb006b81fbe25c232b989677c9dea4e80f3024e WatchSource:0}: Error finding container f36e3fe7651e3b7423e250a30bb006b81fbe25c232b989677c9dea4e80f3024e: Status 404 returned error can't find the container with id f36e3fe7651e3b7423e250a30bb006b81fbe25c232b989677c9dea4e80f3024e
Oct 29 23:12:30.422235 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.422210    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:12:30.431465 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.431441    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:30.431782 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:30.431744    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod916781d5_af66_4d17_ba3f_748bab236eb1.slice/crio-13454da792cd8c93595b1b5ba188b093d14a7a55d808fdb35592464e3466011c WatchSource:0}: Error finding container 13454da792cd8c93595b1b5ba188b093d14a7a55d808fdb35592464e3466011c: Status 404 returned error can't find the container with id 13454da792cd8c93595b1b5ba188b093d14a7a55d808fdb35592464e3466011c
Oct 29 23:12:30.437903 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.437883    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-95nsq"
Oct 29 23:12:30.446864 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.446838    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:12:30.448698 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:30.448618    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc4c683_845b_40f3_ac2e_2c785b221257.slice/crio-cee92baeb3c543a6802e899a973d7a47d97f488c26881ab311903320263e3cb0 WatchSource:0}: Error finding container cee92baeb3c543a6802e899a973d7a47d97f488c26881ab311903320263e3cb0: Status 404 returned error can't find the container with id cee92baeb3c543a6802e899a973d7a47d97f488c26881ab311903320263e3cb0
Oct 29 23:12:30.461396 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:30.461369    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd1e8d6_581b_4e23_89bd_6a9e5854ffa5.slice/crio-fe3ca91ce0d3cfb9bee44ad3ce416052f3d41f3d294ad730c17496a89ee4a0b9 WatchSource:0}: Error finding container fe3ca91ce0d3cfb9bee44ad3ce416052f3d41f3d294ad730c17496a89ee4a0b9: Status 404 returned error can't find the container with id fe3ca91ce0d3cfb9bee44ad3ce416052f3d41f3d294ad730c17496a89ee4a0b9
Oct 29 23:12:30.477297 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.477272    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:30.479565 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.479542    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:12:30.482492 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.482476    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:12:30.489410 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:30.489379    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87b4272a_c0a8_4ff8_8b80_cf9d76552032.slice/crio-2e66982e28329514a8e12f28e3a3da95f22494b7a28b26fb3ec77944d0fdd0f1 WatchSource:0}: Error finding container 2e66982e28329514a8e12f28e3a3da95f22494b7a28b26fb3ec77944d0fdd0f1: Status 404 returned error can't find the container with id 2e66982e28329514a8e12f28e3a3da95f22494b7a28b26fb3ec77944d0fdd0f1
Oct 29 23:12:30.494491 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:30.494232    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeeba32d_22c4_4173_83f0_5b4c391644b0.slice/crio-952140ee592fcad7e773a0943888f1835b17a48d76037ccd3ef8392955128cd9 WatchSource:0}: Error finding container 952140ee592fcad7e773a0943888f1835b17a48d76037ccd3ef8392955128cd9: Status 404 returned error can't find the container with id 952140ee592fcad7e773a0943888f1835b17a48d76037ccd3ef8392955128cd9
Oct 29 23:12:30.498488 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.498292    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:12:30.507081 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:30.507051    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f949f77_5aa6_4443_a09b_6fed361a0d43.slice/crio-b343f5d0aa975e046424b0355a0a0f82af4d1e5d363fed0f128c60fac915fd40 WatchSource:0}: Error finding container b343f5d0aa975e046424b0355a0a0f82af4d1e5d363fed0f128c60fac915fd40: Status 404 returned error can't find the container with id b343f5d0aa975e046424b0355a0a0f82af4d1e5d363fed0f128c60fac915fd40
Oct 29 23:12:30.510305 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:12:30.510279    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54081aac_51a7_47aa_8dab_a04c8d68ab5f.slice/crio-52da2c7c5f90348c938260e89583f55a50e3e9ac6d9223f47e1ca3f8886cefa7 WatchSource:0}: Error finding container 52da2c7c5f90348c938260e89583f55a50e3e9ac6d9223f47e1ca3f8886cefa7: Status 404 returned error can't find the container with id 52da2c7c5f90348c938260e89583f55a50e3e9ac6d9223f47e1ca3f8886cefa7
Oct 29 23:12:30.537713 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.537678    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:12:30.798125 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.798097    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:30.798262 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.798144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:30.798323 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.798264    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:30.798323 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.798292    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:30.798406 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.798334    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs podName:62ec6811-5291-4963-b8f1-52a7ffdc50bc nodeName:}" failed. No retries permitted until 2026-10-29 23:12:31.79831224 +0000 UTC m=+4.137769908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs") pod "network-metrics-daemon-bv5tv" (UID: "62ec6811-5291-4963-b8f1-52a7ffdc50bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:30.798406 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.798351    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret podName:a8ac2ba2-0d4b-4498-a7c6-658412d60ccf nodeName:}" failed. No retries permitted until 2026-10-29 23:12:31.798343407 +0000 UTC m=+4.137801072 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret") pod "global-pull-secret-syncer-wht7s" (UID: "a8ac2ba2-0d4b-4498-a7c6-658412d60ccf") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:30.899596 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:30.899005    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:30.899596 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.899173    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:30.899596 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.899198    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:30.899596 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.899212    2574 projected.go:196] Error preparing data for projected volume kube-api-access-jbssw for pod openshift-network-diagnostics/network-check-target-xhlf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:30.899596 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:30.899267    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw podName:fd1c395e-bc72-4038-80cc-4ab131d96fdb nodeName:}" failed. No retries permitted until 2026-10-29 23:12:31.899247784 +0000 UTC m=+4.238705448 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbssw" (UniqueName: "kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw") pod "network-check-target-xhlf6" (UID: "fd1c395e-bc72-4038-80cc-4ab131d96fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:31.138977 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.138869    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 23:07:30 +0000 UTC" deadline="2028-06-18 12:28:33.348274609 +0000 UTC"
Oct 29 23:12:31.138977 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.138930    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14341h16m2.209349094s"
Oct 29 23:12:31.183675 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.183521    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd" event={"ID":"87b4272a-c0a8-4ff8-8b80-cf9d76552032","Type":"ContainerStarted","Data":"2e66982e28329514a8e12f28e3a3da95f22494b7a28b26fb3ec77944d0fdd0f1"}
Oct 29 23:12:31.188713 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.188634    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-95nsq" event={"ID":"2bc4c683-845b-40f3-ac2e-2c785b221257","Type":"ContainerStarted","Data":"cee92baeb3c543a6802e899a973d7a47d97f488c26881ab311903320263e3cb0"}
Oct 29 23:12:31.192161 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.192039    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nsxh7" event={"ID":"22013da1-6ec7-4b6d-95f8-298e97fb1372","Type":"ContainerStarted","Data":"f36e3fe7651e3b7423e250a30bb006b81fbe25c232b989677c9dea4e80f3024e"}
Oct 29 23:12:31.198345 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.198316    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal" event={"ID":"2f6cf61a946e2452e67c50b8a0479c0d","Type":"ContainerStarted","Data":"b860ef0ce804bd310317385f99bf784bbe458a7663a3b71b327c4b708c5c6f7c"}
Oct 29 23:12:31.202660 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.202605    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vk7hb" event={"ID":"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5","Type":"ContainerStarted","Data":"fe3ca91ce0d3cfb9bee44ad3ce416052f3d41f3d294ad730c17496a89ee4a0b9"}
Oct 29 23:12:31.206908 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.206631    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5z4zk" event={"ID":"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e","Type":"ContainerStarted","Data":"2b01b5effb98e4713c27a97ca01ebf87cb07fc46b39f4caa02ec9fd0bf385741"}
Oct 29 23:12:31.221137 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.221103    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tn9mk" event={"ID":"916781d5-af66-4d17-ba3f-748bab236eb1","Type":"ContainerStarted","Data":"13454da792cd8c93595b1b5ba188b093d14a7a55d808fdb35592464e3466011c"}
Oct 29 23:12:31.237595 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.235270    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmfk5" event={"ID":"54081aac-51a7-47aa-8dab-a04c8d68ab5f","Type":"ContainerStarted","Data":"52da2c7c5f90348c938260e89583f55a50e3e9ac6d9223f47e1ca3f8886cefa7"}
Oct 29 23:12:31.238260 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.238232    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w" event={"ID":"3f949f77-5aa6-4443-a09b-6fed361a0d43","Type":"ContainerStarted","Data":"b343f5d0aa975e046424b0355a0a0f82af4d1e5d363fed0f128c60fac915fd40"}
Oct 29 23:12:31.240786 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.240730    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal" podStartSLOduration=1.24071319 podStartE2EDuration="1.24071319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:31.239901341 +0000 UTC m=+3.579359040" watchObservedRunningTime="2026-10-29 23:12:31.24071319 +0000 UTC m=+3.580170878"
Oct 29 23:12:31.244854 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.244766    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n4lkv" event={"ID":"eeeba32d-22c4-4173-83f0-5b4c391644b0","Type":"ContainerStarted","Data":"952140ee592fcad7e773a0943888f1835b17a48d76037ccd3ef8392955128cd9"}
Oct 29 23:12:31.805759 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.805488    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:31.805956 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.805818    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:31.806024 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:31.805996    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:31.806075 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:31.806068    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret podName:a8ac2ba2-0d4b-4498-a7c6-658412d60ccf nodeName:}" failed. No retries permitted until 2026-10-29 23:12:33.806046988 +0000 UTC m=+6.145504653 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret") pod "global-pull-secret-syncer-wht7s" (UID: "a8ac2ba2-0d4b-4498-a7c6-658412d60ccf") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:31.806503 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:31.806482    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:31.806557 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:31.806549    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs podName:62ec6811-5291-4963-b8f1-52a7ffdc50bc nodeName:}" failed. No retries permitted until 2026-10-29 23:12:33.806530406 +0000 UTC m=+6.145988074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs") pod "network-metrics-daemon-bv5tv" (UID: "62ec6811-5291-4963-b8f1-52a7ffdc50bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:31.907594 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:31.906869    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:31.907594 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:31.907054    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:31.907594 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:31.907073    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:31.907594 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:31.907087    2574 projected.go:196] Error preparing data for projected volume kube-api-access-jbssw for pod openshift-network-diagnostics/network-check-target-xhlf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:31.907594 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:31.907147    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw podName:fd1c395e-bc72-4038-80cc-4ab131d96fdb nodeName:}" failed. No retries permitted until 2026-10-29 23:12:33.907127291 +0000 UTC m=+6.246584953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbssw" (UniqueName: "kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw") pod "network-check-target-xhlf6" (UID: "fd1c395e-bc72-4038-80cc-4ab131d96fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:32.140097 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:32.140022    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 23:07:30 +0000 UTC" deadline="2028-07-03 07:45:45.392894907 +0000 UTC"
Oct 29 23:12:32.140624 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:32.140112    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14696h33m13.252788326s"
Oct 29 23:12:32.146991 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:32.146961    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:32.147137 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:32.147112    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:32.147893 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:32.147604    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:32.147893 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:32.147717    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:32.148579 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:32.148272    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:32.148579 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:32.148385    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:32.263152 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:32.262707    2574 generic.go:356] "Generic (PLEG): container finished" podID="aa79b04c65fcc2c4af3f0c1b29fc1ed6" containerID="2f656a59facaba97fbabae29c0751b97aff91da38d1bafb0db437d74f5a1848e" exitCode=0
Oct 29 23:12:32.263152 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:32.262829    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal" event={"ID":"aa79b04c65fcc2c4af3f0c1b29fc1ed6","Type":"ContainerDied","Data":"2f656a59facaba97fbabae29c0751b97aff91da38d1bafb0db437d74f5a1848e"}
Oct 29 23:12:33.284914 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:33.284793    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal" event={"ID":"aa79b04c65fcc2c4af3f0c1b29fc1ed6","Type":"ContainerStarted","Data":"3a36e5d7e20a7429043cf2e21dbb65cea88ab146a3715760f3171018fda1d764"}
Oct 29 23:12:33.823145 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:33.823111    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:33.823371 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:33.823197    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:33.823371 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:33.823333    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:33.823489 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:33.823400    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs podName:62ec6811-5291-4963-b8f1-52a7ffdc50bc nodeName:}" failed. No retries permitted until 2026-10-29 23:12:37.823379714 +0000 UTC m=+10.162837387 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs") pod "network-metrics-daemon-bv5tv" (UID: "62ec6811-5291-4963-b8f1-52a7ffdc50bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:33.823851 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:33.823830    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:33.823951 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:33.823919    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret podName:a8ac2ba2-0d4b-4498-a7c6-658412d60ccf nodeName:}" failed. No retries permitted until 2026-10-29 23:12:37.823874805 +0000 UTC m=+10.163332466 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret") pod "global-pull-secret-syncer-wht7s" (UID: "a8ac2ba2-0d4b-4498-a7c6-658412d60ccf") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:33.924600 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:33.924555    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:33.924784 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:33.924727    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:33.924784 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:33.924750    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:33.924784 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:33.924763    2574 projected.go:196] Error preparing data for projected volume kube-api-access-jbssw for pod openshift-network-diagnostics/network-check-target-xhlf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:33.925045 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:33.924840    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw podName:fd1c395e-bc72-4038-80cc-4ab131d96fdb nodeName:}" failed. No retries permitted until 2026-10-29 23:12:37.924820812 +0000 UTC m=+10.264278486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbssw" (UniqueName: "kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw") pod "network-check-target-xhlf6" (UID: "fd1c395e-bc72-4038-80cc-4ab131d96fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:34.151909 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:34.151832    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:34.152081 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:34.151966    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:34.152393 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:34.152375    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:34.152521 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:34.152499    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:34.152592 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:34.152553    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:34.152652 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:34.152623    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:34.293525 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:34.293487    2574 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="d056999b16e340a2858c82fa1a599b9f88211e4f1886edfadb38e2e27c74382a" exitCode=0
Oct 29 23:12:34.294551 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:34.294527    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmfk5" event={"ID":"54081aac-51a7-47aa-8dab-a04c8d68ab5f","Type":"ContainerDied","Data":"d056999b16e340a2858c82fa1a599b9f88211e4f1886edfadb38e2e27c74382a"}
Oct 29 23:12:34.344520 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:34.344457    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal" podStartSLOduration=4.344436544 podStartE2EDuration="4.344436544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:33.328677422 +0000 UTC m=+5.668135106" watchObservedRunningTime="2026-10-29 23:12:34.344436544 +0000 UTC m=+6.683894229"
Oct 29 23:12:35.302558 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:35.301878    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5z4zk" event={"ID":"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e","Type":"ContainerStarted","Data":"192c5caa5ec53fc5b994b9114cb7d1da6501c5e39f20c63781898c06b2be976e"}
Oct 29 23:12:36.146170 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:36.146090    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:36.146346 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:36.146247    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:36.146703 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:36.146679    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:36.146837 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:36.146795    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:36.146950 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:36.146882    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:36.147054 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:36.146955    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:36.309840 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:36.309793    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tn9mk" event={"ID":"916781d5-af66-4d17-ba3f-748bab236eb1","Type":"ContainerStarted","Data":"307c80bb63e84e8ab97ce3c8c24c3a37b3a625d4df90ecad9944d8b06fb1973c"}
Oct 29 23:12:36.353343 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:36.353283    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5z4zk" podStartSLOduration=3.451755401 podStartE2EDuration="7.353261305s" totalImagesPullingTime="3.901505904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:35.346684825 +0000 UTC m=+7.686142512" watchObservedRunningTime="2026-10-29 23:12:36.353261305 +0000 UTC m=+8.692718992"
Oct 29 23:12:36.550442 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:36.550346    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:36.551181 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:36.551161    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:36.595403 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:36.595339    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tn9mk" podStartSLOduration=2.378877611 podStartE2EDuration="7.595320233s" totalImagesPullingTime="5.216442622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:36.352891684 +0000 UTC m=+8.692349370" watchObservedRunningTime="2026-10-29 23:12:36.595320233 +0000 UTC m=+8.934777918"
Oct 29 23:12:37.318606 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:37.318565    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w" event={"ID":"3f949f77-5aa6-4443-a09b-6fed361a0d43","Type":"ContainerStarted","Data":"0b66b0dc18ed8fcd3d833de62865df2817576d761c36e8fe58dd940f401fe82e"}
Oct 29 23:12:37.859960 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:37.859917    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:37.860131 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:37.859979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:37.860131 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:37.860116    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:37.860234 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:37.860179    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret podName:a8ac2ba2-0d4b-4498-a7c6-658412d60ccf nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.860158773 +0000 UTC m=+18.199616451 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret") pod "global-pull-secret-syncer-wht7s" (UID: "a8ac2ba2-0d4b-4498-a7c6-658412d60ccf") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:37.860670 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:37.860636    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:37.860803 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:37.860698    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs podName:62ec6811-5291-4963-b8f1-52a7ffdc50bc nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.860681633 +0000 UTC m=+18.200139311 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs") pod "network-metrics-daemon-bv5tv" (UID: "62ec6811-5291-4963-b8f1-52a7ffdc50bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:37.960586 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:37.960553    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:37.960735 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:37.960713    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:37.960820 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:37.960748    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:37.960820 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:37.960764    2574 projected.go:196] Error preparing data for projected volume kube-api-access-jbssw for pod openshift-network-diagnostics/network-check-target-xhlf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:37.960940 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:37.960845    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw podName:fd1c395e-bc72-4038-80cc-4ab131d96fdb nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.960824355 +0000 UTC m=+18.300282034 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbssw" (UniqueName: "kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw") pod "network-check-target-xhlf6" (UID: "fd1c395e-bc72-4038-80cc-4ab131d96fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:38.133228 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:38.133022    2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 29 23:12:38.146193 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:38.146163    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:38.146320 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:38.146303    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:38.146447 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:38.146397    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:38.146506 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:38.146479    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:38.147741 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:38.147716    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:38.148001 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:38.147812    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:38.152655 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:38.152552    2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-29T23:12:38.13304125Z","UUID":"35418983-b9b6-4d5a-b14a-110a213e080b","Handler":null,"Name":"","Endpoint":""}
Oct 29 23:12:38.154325 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:38.154299    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 29 23:12:38.154434 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:38.154347    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 29 23:12:38.325774 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:38.325395    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:12:38.326266 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:38.325861    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w" event={"ID":"3f949f77-5aa6-4443-a09b-6fed361a0d43","Type":"ContainerStarted","Data":"59d2248d8165d71b4c206daeb8d144e1775e6789b41def3f4efd56e7f8cc64d1"}
Oct 29 23:12:39.332651 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:39.332609    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n4lkv" event={"ID":"eeeba32d-22c4-4173-83f0-5b4c391644b0","Type":"ContainerStarted","Data":"0726b115e5394c226c789511ab9efc8b077ca7b4e80dddee22099b681c01d551"}
Oct 29 23:12:39.340885 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:39.340848    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-95nsq" event={"ID":"2bc4c683-845b-40f3-ac2e-2c785b221257","Type":"ContainerStarted","Data":"c2d4fc8ca4702ea2aa458fd6c52260f0a30285757b1c5cc9852c5167235a31d8"}
Oct 29 23:12:39.344602 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:39.344570    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmfk5" event={"ID":"54081aac-51a7-47aa-8dab-a04c8d68ab5f","Type":"ContainerStarted","Data":"8ab70ab24c46b716274331f6a97f6a6a91e6b62362fd441370f327291212db0d"}
Oct 29 23:12:39.424753 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:39.424694    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-95nsq" podStartSLOduration=2.153934632 podStartE2EDuration="10.424664917s" totalImagesPullingTime="8.270730285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:39.424489669 +0000 UTC m=+11.763947355" watchObservedRunningTime="2026-10-29 23:12:39.424664917 +0000 UTC m=+11.764122618"
Oct 29 23:12:39.425169 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:39.425137    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-n4lkv" podStartSLOduration=2.27415857 podStartE2EDuration="10.425127073s" totalImagesPullingTime="8.150968503s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:39.379372223 +0000 UTC m=+11.718829906" watchObservedRunningTime="2026-10-29 23:12:39.425127073 +0000 UTC m=+11.764584756"
Oct 29 23:12:40.147247 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:40.147013    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:40.147406 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:40.147119    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:40.147406 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:40.147359    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:40.147406 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:40.147130    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:40.147553 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:40.147424    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:40.147553 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:40.147490    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:40.348919 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:40.348883    2574 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="8ab70ab24c46b716274331f6a97f6a6a91e6b62362fd441370f327291212db0d" exitCode=0
Oct 29 23:12:40.349351 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:40.348969    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmfk5" event={"ID":"54081aac-51a7-47aa-8dab-a04c8d68ab5f","Type":"ContainerDied","Data":"8ab70ab24c46b716274331f6a97f6a6a91e6b62362fd441370f327291212db0d"}
Oct 29 23:12:40.351670 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:40.351648    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w" event={"ID":"3f949f77-5aa6-4443-a09b-6fed361a0d43","Type":"ContainerStarted","Data":"1af2258ebc6ee1b1c4df0643b36496ac57ec7c36e3954ac7f11848a4d3b29d07"}
Oct 29 23:12:40.440265 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:40.440199    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w" podStartSLOduration=3.057007008 podStartE2EDuration="11.440178328s" totalImagesPullingTime="8.38317132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:40.439842882 +0000 UTC m=+12.779300563" watchObservedRunningTime="2026-10-29 23:12:40.440178328 +0000 UTC m=+12.779636010"
Oct 29 23:12:41.355280 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:41.355244    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nsxh7" event={"ID":"22013da1-6ec7-4b6d-95f8-298e97fb1372","Type":"ContainerStarted","Data":"5eec2c9dfc18e30fbfba97121d39241d3560a83d576cf26574875321833c9128"}
Oct 29 23:12:41.356671 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:41.356644    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vk7hb" event={"ID":"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5","Type":"ContainerStarted","Data":"2f02815be1fa3dd62d944b0d4e820b299ca5ba6280c216708ea5d0ff7a7f2693"}
Oct 29 23:12:41.406804 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:41.406237    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nsxh7" podStartSLOduration=1.62606329 podStartE2EDuration="12.406220151s" totalImagesPullingTime="10.780156861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:41.404841398 +0000 UTC m=+13.744299072" watchObservedRunningTime="2026-10-29 23:12:41.406220151 +0000 UTC m=+13.745677832"
Oct 29 23:12:41.446251 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:41.446208    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vk7hb" podStartSLOduration=3.952675491 podStartE2EDuration="12.446196062s" totalImagesPullingTime="8.493520571s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:41.446074489 +0000 UTC m=+13.785532168" watchObservedRunningTime="2026-10-29 23:12:41.446196062 +0000 UTC m=+13.785653741"
Oct 29 23:12:42.145579 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:42.145550    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:42.145579 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:42.145566    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:42.145836 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:42.145666    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:42.145836 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:42.145706    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:42.145836 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:42.145774    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:42.145992 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:42.145896    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:42.360774 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:42.360725    2574 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="f7114c7a00efda02498591754056d2e74a509333ef6865718fc67a9f5d6933f8" exitCode=0
Oct 29 23:12:42.361347 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:42.360782    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmfk5" event={"ID":"54081aac-51a7-47aa-8dab-a04c8d68ab5f","Type":"ContainerDied","Data":"f7114c7a00efda02498591754056d2e74a509333ef6865718fc67a9f5d6933f8"}
Oct 29 23:12:43.366528 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:43.366492    2574 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="97e76f94125d393d0ed0187f84501998d3badc0f55990fe0ad69ebe5d8a37daf" exitCode=0
Oct 29 23:12:43.367065 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:43.366552    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmfk5" event={"ID":"54081aac-51a7-47aa-8dab-a04c8d68ab5f","Type":"ContainerDied","Data":"97e76f94125d393d0ed0187f84501998d3badc0f55990fe0ad69ebe5d8a37daf"}
Oct 29 23:12:44.146036 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:44.145817    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:44.146213 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:44.145817    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:44.146213 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:44.146151    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:44.146213 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:44.145875    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:44.146331 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:44.146207    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:44.146331 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:44.146300    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:45.921358 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:45.921335    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:45.921811 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:45.921376    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:45.921811 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:45.921455    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:45.921811 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:45.921474    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:45.921811 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:45.921506    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs podName:62ec6811-5291-4963-b8f1-52a7ffdc50bc nodeName:}" failed. No retries permitted until 2026-10-29 23:13:01.921489215 +0000 UTC m=+34.260946889 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs") pod "network-metrics-daemon-bv5tv" (UID: "62ec6811-5291-4963-b8f1-52a7ffdc50bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:45.921811 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:45.921526    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret podName:a8ac2ba2-0d4b-4498-a7c6-658412d60ccf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:01.921515789 +0000 UTC m=+34.260973456 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret") pod "global-pull-secret-syncer-wht7s" (UID: "a8ac2ba2-0d4b-4498-a7c6-658412d60ccf") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:46.022579 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.022540    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:46.022747 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:46.022714    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:46.022818 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:46.022757    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:46.022818 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:46.022772    2574 projected.go:196] Error preparing data for projected volume kube-api-access-jbssw for pod openshift-network-diagnostics/network-check-target-xhlf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:46.022928 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:46.022841    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw podName:fd1c395e-bc72-4038-80cc-4ab131d96fdb nodeName:}" failed. No retries permitted until 2026-10-29 23:13:02.022814459 +0000 UTC m=+34.362272116 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbssw" (UniqueName: "kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw") pod "network-check-target-xhlf6" (UID: "fd1c395e-bc72-4038-80cc-4ab131d96fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:46.146470 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.146289    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:46.146651 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.146373    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:46.146651 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:46.146548    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:46.146651 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.146382    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:46.146651 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:46.146614    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:46.146888 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:46.146676    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:46.375512 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.375423    2574 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="bd3acb832876de97996dffe57f5fa336b66f2b0c926bd9470adab47842cd3c1f" exitCode=0
Oct 29 23:12:46.375512 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.375464    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmfk5" event={"ID":"54081aac-51a7-47aa-8dab-a04c8d68ab5f","Type":"ContainerDied","Data":"bd3acb832876de97996dffe57f5fa336b66f2b0c926bd9470adab47842cd3c1f"}
Oct 29 23:12:46.378473 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.378452    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd" event={"ID":"87b4272a-c0a8-4ff8-8b80-cf9d76552032","Type":"ContainerStarted","Data":"2424c5623636c14367591c4a54e5022096c41a0ba3fdea49626627d05309a8fa"}
Oct 29 23:12:46.378534 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.378509    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd" event={"ID":"87b4272a-c0a8-4ff8-8b80-cf9d76552032","Type":"ContainerStarted","Data":"e8b3d869ec12e6f676937f54f922f842c75cc4a0f0a1d1fbc85facf34761625a"}
Oct 29 23:12:46.378565 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.378543    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd" event={"ID":"87b4272a-c0a8-4ff8-8b80-cf9d76552032","Type":"ContainerStarted","Data":"0fac4179843c4ebb228d0900c9edf504b81b0a4dab92c7af05353edf26c38d9b"}
Oct 29 23:12:46.378597 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.378574    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd" event={"ID":"87b4272a-c0a8-4ff8-8b80-cf9d76552032","Type":"ContainerStarted","Data":"71e07baf411bc9c720c415a2ad017510a557bd3640eab4e07f9f40c1aad1e519"}
Oct 29 23:12:46.378625 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.378600    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd" event={"ID":"87b4272a-c0a8-4ff8-8b80-cf9d76552032","Type":"ContainerStarted","Data":"dd3ab9705cf7927d1f085ab1ef0b072fdccdc9959d4ae44e6242426ade855b25"}
Oct 29 23:12:46.378658 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:46.378633    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd" event={"ID":"87b4272a-c0a8-4ff8-8b80-cf9d76552032","Type":"ContainerStarted","Data":"e351784eac398437ab9fa24995ba981990a29a71de658ad52bc261ea999f44a4"}
Oct 29 23:12:47.383481 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:47.383446    2574 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="ea993ffdfadf7e2a1281f3d4c035b8632c3a02d3eb8829d9e72bc85c2f9777e7" exitCode=0
Oct 29 23:12:47.383867 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:47.383488    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmfk5" event={"ID":"54081aac-51a7-47aa-8dab-a04c8d68ab5f","Type":"ContainerDied","Data":"ea993ffdfadf7e2a1281f3d4c035b8632c3a02d3eb8829d9e72bc85c2f9777e7"}
Oct 29 23:12:48.146705 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:48.146673    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:48.146897 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:48.146781    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:48.146897 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:48.146835    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:48.146897 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:48.146831    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:48.147047 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:48.146918    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:48.147047 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:48.147026    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:48.389672 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:48.389581    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmfk5" event={"ID":"54081aac-51a7-47aa-8dab-a04c8d68ab5f","Type":"ContainerStarted","Data":"ebb18ebcf3d86d631346265ac3bd43b2daa84f1f41d8a28625a4ee2d32188b59"}
Oct 29 23:12:48.392715 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:48.392690    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd" event={"ID":"87b4272a-c0a8-4ff8-8b80-cf9d76552032","Type":"ContainerStarted","Data":"9b40408e63271fbebe31dc72af8e7dc4aa8f2d1350065ce4a81edcc4c2257262"}
Oct 29 23:12:48.462873 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:48.461628    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wmfk5" podStartSLOduration=7.7058895750000005 podStartE2EDuration="19.461603566s" totalImagesPullingTime="11.755713991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:48.46034506 +0000 UTC m=+20.799802742" watchObservedRunningTime="2026-10-29 23:12:48.461603566 +0000 UTC m=+20.801061247"
Oct 29 23:12:50.007390 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:50.007359    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n4lkv_eeeba32d-22c4-4173-83f0-5b4c391644b0/dns-node-resolver/0.log"
Oct 29 23:12:50.145958 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:50.145928    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:50.146061 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:50.145935    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:50.146061 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:50.146033    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:50.146061 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:50.146041    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:50.146209 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:50.146115    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:50.146209 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:50.146178    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:50.400988 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:50.400912    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd" event={"ID":"87b4272a-c0a8-4ff8-8b80-cf9d76552032","Type":"ContainerStarted","Data":"a905972437da8f81a1ad142047f316d7b362b2ea7b1cac03077ccbfb7a33e9ad"}
Oct 29 23:12:50.401203 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:50.401181    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:50.401265 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:50.401252    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:50.416402 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:50.416373    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:50.490929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:50.490710    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd" podStartSLOduration=6.276812429 podStartE2EDuration="21.49069738s" totalImagesPullingTime="15.213884951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:50.490495509 +0000 UTC m=+22.829953189" watchObservedRunningTime="2026-10-29 23:12:50.49069738 +0000 UTC m=+22.830155061"
Oct 29 23:12:51.339678 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:51.339653    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tn9mk_916781d5-af66-4d17-ba3f-748bab236eb1/node-ca/0.log"
Oct 29 23:12:51.403718 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:51.403689    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:51.418707 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:51.418683    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:12:52.146330 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:52.146294    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:52.146570 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:52.146359    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:52.146570 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:52.146366    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:52.146570 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:52.146463    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:52.146687 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:52.146569    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:52.146687 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:52.146638    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:54.145723 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:54.145686    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:54.146221 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:54.145809    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:54.146221 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:54.145810    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:54.146221 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:54.145836    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:54.146221 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:54.145880    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:54.146221 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:54.145923    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:56.146546 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:56.146513    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:56.147024 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:56.146513    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:56.147024 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:56.146620    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:56.147024 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:56.146520    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:56.147024 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:56.146701    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:56.147024 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:56.146815    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:12:57.331790 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:57.331754    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:57.332251 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:57.331911    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:12:57.332351 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:57.332333    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:12:58.146526 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:58.146492    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:12:58.146709 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:58.146584    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:12:58.146709 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:58.146625    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:12:58.146709 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:58.146646    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:12:58.146709 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:12:58.146679    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:12:58.146934 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:12:58.146753    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:13:00.146505 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:00.146475    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:00.146951 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:00.146508    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:00.146951 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:00.146568    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:13:00.146951 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:00.146627    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:13:00.146951 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:00.146674    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:00.146951 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:00.146774    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:13:01.937349 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:01.937313    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:01.937349 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:01.937356    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:01.937831 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:01.937454    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:13:01.937831 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:01.937457    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:13:01.937831 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:01.937505    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret podName:a8ac2ba2-0d4b-4498-a7c6-658412d60ccf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:33.937492576 +0000 UTC m=+66.276950248 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret") pod "global-pull-secret-syncer-wht7s" (UID: "a8ac2ba2-0d4b-4498-a7c6-658412d60ccf") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:13:01.937831 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:01.937520    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs podName:62ec6811-5291-4963-b8f1-52a7ffdc50bc nodeName:}" failed. No retries permitted until 2026-10-29 23:13:33.937513627 +0000 UTC m=+66.276971285 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs") pod "network-metrics-daemon-bv5tv" (UID: "62ec6811-5291-4963-b8f1-52a7ffdc50bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:13:02.038210 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:02.038176    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:02.038372 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:02.038300    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:13:02.038372 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:02.038317    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:13:02.038372 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:02.038329    2574 projected.go:196] Error preparing data for projected volume kube-api-access-jbssw for pod openshift-network-diagnostics/network-check-target-xhlf6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:13:02.038487 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:02.038374    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw podName:fd1c395e-bc72-4038-80cc-4ab131d96fdb nodeName:}" failed. No retries permitted until 2026-10-29 23:13:34.038362019 +0000 UTC m=+66.377819689 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbssw" (UniqueName: "kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw") pod "network-check-target-xhlf6" (UID: "fd1c395e-bc72-4038-80cc-4ab131d96fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:13:02.146441 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:02.146408    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:02.146441 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:02.146423    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:02.146441 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:02.146429    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:02.146709 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:02.146543    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:13:02.146709 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:02.146643    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:13:02.146847 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:02.146706    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:13:04.146108 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:04.146062    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:04.146108 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:04.146113    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:04.146672 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:04.146078    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:04.146672 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:04.146187    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:13:04.146672 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:04.146271    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:13:04.146672 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:04.146332    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:13:06.145866 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:06.145834    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:06.145866 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:06.145852    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:06.146371 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:06.145956    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:13:06.146371 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:06.146025    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:06.146371 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:06.146020    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:13:06.146371 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:06.146089    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:13:08.148305 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:08.148268    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:08.148809 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:08.148379    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:08.148809 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:08.148426    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:08.148809 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:08.148534    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:13:08.148809 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:08.148778    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:13:08.149055 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:08.149033    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:13:10.145869 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:10.145839    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:10.146325 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:10.145979    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:13:10.146325 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:10.146203    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:10.146325 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:10.146286    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:13:10.146486 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:10.146439    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:10.146539 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:10.146517    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:13:10.232449 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:10.232419    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xhlf6"]
Oct 29 23:13:10.235241 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:10.235214    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wht7s"]
Oct 29 23:13:10.236046 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:10.236025    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bv5tv"]
Oct 29 23:13:10.441254 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:10.441227    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:10.441254 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:10.441240    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:10.441458 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:10.441329    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:10.441458 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:10.441345    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:13:10.441458 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:10.441415    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:13:10.441551 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:10.441479    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:13:10.993375 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:10.993351    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n4lkv_eeeba32d-22c4-4173-83f0-5b4c391644b0/dns-node-resolver/0.log"
Oct 29 23:13:12.146190 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:12.146157    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:12.146722 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:12.146157    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:12.146722 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:12.146280    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:13:12.146722 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:12.146163    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:12.146722 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:12.146352    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:13:12.146722 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:12.146416    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:13:12.589433 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:12.589227    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tn9mk_916781d5-af66-4d17-ba3f-748bab236eb1/node-ca/0.log"
Oct 29 23:13:14.146128 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:14.146096    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:14.146128 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:14.146112    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:14.146547 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:14.146214    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xhlf6" podUID="fd1c395e-bc72-4038-80cc-4ab131d96fdb"
Oct 29 23:13:14.146547 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:14.146275    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:14.146547 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:14.146355    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bv5tv" podUID="62ec6811-5291-4963-b8f1-52a7ffdc50bc"
Oct 29 23:13:14.146547 ip-10-0-138-189 kubenswrapper[2574]: E1029 23:13:14.146392    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wht7s" podUID="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf"
Oct 29 23:13:15.456144 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.456119    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeReady"
Oct 29 23:13:15.456524 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.456236    2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 29 23:13:15.530905 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.530870    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fk7ml"]
Oct 29 23:13:15.533970 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.533953    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.537667 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.537643    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qlpqj"]
Oct 29 23:13:15.540508 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.540489    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:15.552926 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.552906    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 29 23:13:15.553004 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.552960    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 29 23:13:15.553054 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.553026    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 29 23:13:15.553090 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.553067    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 29 23:13:15.553143 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.553130    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-88spp\""
Oct 29 23:13:15.559017 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.559004    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-rlkhk\""
Oct 29 23:13:15.565651 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.565628    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qlpqj"]
Oct 29 23:13:15.566406 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.566385    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fk7ml"]
Oct 29 23:13:15.636483 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.636457    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4ef6497-03ed-4ed8-beb9-1cde1f435655-data-volume\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.636633 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.636494    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4ef6497-03ed-4ed8-beb9-1cde1f435655-crio-socket\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.636633 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.636513    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfhnw\" (UniqueName: \"kubernetes.io/projected/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-api-access-tfhnw\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.636633 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.636531    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.636732 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.636651    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4ef6497-03ed-4ed8-beb9-1cde1f435655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.637166 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.637148    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mnl8f"]
Oct 29 23:13:15.640053 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.640039    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.653865 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.653842    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 29 23:13:15.653996 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.653925    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 29 23:13:15.653996 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.653925    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 29 23:13:15.661899 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.661874    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mnl8f"]
Oct 29 23:13:15.737981 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.737900    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4ef6497-03ed-4ed8-beb9-1cde1f435655-data-volume\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.737981 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.737935    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4ef6497-03ed-4ed8-beb9-1cde1f435655-crio-socket\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.737981 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.737956    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfhnw\" (UniqueName: \"kubernetes.io/projected/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-api-access-tfhnw\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.738223 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.738040    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.738223 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.738092    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-cert\") pod \"ingress-canary-qlpqj\" (UID: \"9eb8b8c3-aae3-4116-9d9e-244d34ba0271\") " pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:15.738223 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.738187    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c5z9\" (UniqueName: \"kubernetes.io/projected/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-kube-api-access-8c5z9\") pod \"ingress-canary-qlpqj\" (UID: \"9eb8b8c3-aae3-4116-9d9e-244d34ba0271\") " pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:15.738223 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.738211    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4ef6497-03ed-4ed8-beb9-1cde1f435655-crio-socket\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.738364 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.738246    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4ef6497-03ed-4ed8-beb9-1cde1f435655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.738364 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.738288    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4ef6497-03ed-4ed8-beb9-1cde1f435655-data-volume\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.738550 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.738526    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.741851 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.741829    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4ef6497-03ed-4ed8-beb9-1cde1f435655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.752788 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.752771    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 29 23:13:15.752883 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.752796    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cxqm7\""
Oct 29 23:13:15.753360 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.753349    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 29 23:13:15.762602 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.762576    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfhnw\" (UniqueName: \"kubernetes.io/projected/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-api-access-tfhnw\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.839302 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.839270    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/802fe00a-6d8f-483e-9fb5-2ea266688ad1-config-volume\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.839302 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.839305    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/802fe00a-6d8f-483e-9fb5-2ea266688ad1-tmp-dir\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.839473 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.839342    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-cert\") pod \"ingress-canary-qlpqj\" (UID: \"9eb8b8c3-aae3-4116-9d9e-244d34ba0271\") " pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:15.839473 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.839396    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c5z9\" (UniqueName: \"kubernetes.io/projected/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-kube-api-access-8c5z9\") pod \"ingress-canary-qlpqj\" (UID: \"9eb8b8c3-aae3-4116-9d9e-244d34ba0271\") " pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:15.839473 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.839436    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/802fe00a-6d8f-483e-9fb5-2ea266688ad1-metrics-tls\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.839566 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.839490    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwqjz\" (UniqueName: \"kubernetes.io/projected/802fe00a-6d8f-483e-9fb5-2ea266688ad1-kube-api-access-gwqjz\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.841562 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.841540    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-cert\") pod \"ingress-canary-qlpqj\" (UID: \"9eb8b8c3-aae3-4116-9d9e-244d34ba0271\") " pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:15.850427 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.850408    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:15.862933 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.862910    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c5z9\" (UniqueName: \"kubernetes.io/projected/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-kube-api-access-8c5z9\") pod \"ingress-canary-qlpqj\" (UID: \"9eb8b8c3-aae3-4116-9d9e-244d34ba0271\") " pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:15.940239 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.940205    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/802fe00a-6d8f-483e-9fb5-2ea266688ad1-metrics-tls\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.940379 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.940255    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwqjz\" (UniqueName: \"kubernetes.io/projected/802fe00a-6d8f-483e-9fb5-2ea266688ad1-kube-api-access-gwqjz\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.940379 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.940284    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/802fe00a-6d8f-483e-9fb5-2ea266688ad1-config-volume\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.940379 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.940303    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/802fe00a-6d8f-483e-9fb5-2ea266688ad1-tmp-dir\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.940792 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.940760    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/802fe00a-6d8f-483e-9fb5-2ea266688ad1-tmp-dir\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.940913 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.940828    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/802fe00a-6d8f-483e-9fb5-2ea266688ad1-config-volume\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.942359 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.942342    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/802fe00a-6d8f-483e-9fb5-2ea266688ad1-metrics-tls\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:15.963825 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:15.963803    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwqjz\" (UniqueName: \"kubernetes.io/projected/802fe00a-6d8f-483e-9fb5-2ea266688ad1-kube-api-access-gwqjz\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:16.146023 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.145989    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:16.146188 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.145983    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:16.146188 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.146005    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:16.152172 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.152150    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:16.174590 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.174564    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fk7ml"]
Oct 29 23:13:16.174694 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.174573    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 29 23:13:16.177316 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.177295    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-m6t5c\""
Oct 29 23:13:16.177529 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.177512    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wscf4\""
Oct 29 23:13:16.177600 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.177519    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-z28mk\""
Oct 29 23:13:16.177600 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.177519    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 29 23:13:16.177600 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.177549    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 29 23:13:16.252934 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.252910    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:16.273502 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.273457    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 29 23:13:16.451131 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.451106    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qlpqj"]
Oct 29 23:13:16.451935 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:13:16.451912    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb8b8c3_aae3_4116_9d9e_244d34ba0271.slice/crio-2998256652bea035ba14fa892c488c98f5d860a3edf9ddb465ebf694608034b8 WatchSource:0}: Error finding container 2998256652bea035ba14fa892c488c98f5d860a3edf9ddb465ebf694608034b8: Status 404 returned error can't find the container with id 2998256652bea035ba14fa892c488c98f5d860a3edf9ddb465ebf694608034b8
Oct 29 23:13:16.456515 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.456487    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qlpqj" event={"ID":"9eb8b8c3-aae3-4116-9d9e-244d34ba0271","Type":"ContainerStarted","Data":"2998256652bea035ba14fa892c488c98f5d860a3edf9ddb465ebf694608034b8"}
Oct 29 23:13:16.457905 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.457881    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fk7ml" event={"ID":"f4ef6497-03ed-4ed8-beb9-1cde1f435655","Type":"ContainerStarted","Data":"c7a4e29a07883dffa369727bb927756d505eae353ed9ca05b00ca216a7f7354a"}
Oct 29 23:13:16.457980 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.457929    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fk7ml" event={"ID":"f4ef6497-03ed-4ed8-beb9-1cde1f435655","Type":"ContainerStarted","Data":"0c222a2193223b38a930acaf812a9a17d91987fc97b53582e03319c9b9109770"}
Oct 29 23:13:16.562324 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.562295    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mnl8f"]
Oct 29 23:13:16.565284 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:13:16.565256    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod802fe00a_6d8f_483e_9fb5_2ea266688ad1.slice/crio-6b450ddcc054ff72ae223cb160cfaf267a5093104ac64e78998c8ac0553345f3 WatchSource:0}: Error finding container 6b450ddcc054ff72ae223cb160cfaf267a5093104ac64e78998c8ac0553345f3: Status 404 returned error can't find the container with id 6b450ddcc054ff72ae223cb160cfaf267a5093104ac64e78998c8ac0553345f3
Oct 29 23:13:16.582392 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.582369    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dn5kk"]
Oct 29 23:13:16.585520 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.585500    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.639590 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.639214    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 29 23:13:16.639590 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.639249    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 29 23:13:16.639590 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.639294    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 29 23:13:16.639590 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.639366    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 29 23:13:16.644658 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.644636    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-accelerators-collector-config\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.644855 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.644675    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-root\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.644855 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.644694    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-tls\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.644964 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.644867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-metrics-client-ca\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.644964 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.644909    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-textfile\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.645032 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.644960    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-wtmp\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.645032 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.645001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-sys\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.645032 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.645025    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.645144 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.645043    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlpjn\" (UniqueName: \"kubernetes.io/projected/59d798e8-802a-4a6a-8f5a-063d69972c13-kube-api-access-vlpjn\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.675385 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.675354    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 29 23:13:16.675543 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.675415    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 29 23:13:16.675543 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.675452    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8hh8s\""
Oct 29 23:13:16.745821 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.745758    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlpjn\" (UniqueName: \"kubernetes.io/projected/59d798e8-802a-4a6a-8f5a-063d69972c13-kube-api-access-vlpjn\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.745821 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.745809    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-accelerators-collector-config\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746021 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.745846    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-root\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746021 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.745862    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-tls\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746021 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.745903    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-metrics-client-ca\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746021 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.745936    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-textfile\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746021 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.745958    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-root\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746021 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.745956    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-wtmp\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746341 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.746026    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-sys\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746341 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.746052    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746341 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.746234    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-wtmp\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746341 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.746288    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-sys\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746341 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.746330    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-textfile\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746599 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.746486    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-accelerators-collector-config\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.746599 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.746562    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-metrics-client-ca\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.748396 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.748372    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.748504 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.748424    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-tls\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.770163 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.770129    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlpjn\" (UniqueName: \"kubernetes.io/projected/59d798e8-802a-4a6a-8f5a-063d69972c13-kube-api-access-vlpjn\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.902127 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:16.902085    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:16.913689 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:13:16.913658    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d798e8_802a_4a6a_8f5a_063d69972c13.slice/crio-76edbc97b64d77cc4eb3bd88ad6e43e5ab4f389627af1d1d293c28c93aeb7d4b WatchSource:0}: Error finding container 76edbc97b64d77cc4eb3bd88ad6e43e5ab4f389627af1d1d293c28c93aeb7d4b: Status 404 returned error can't find the container with id 76edbc97b64d77cc4eb3bd88ad6e43e5ab4f389627af1d1d293c28c93aeb7d4b
Oct 29 23:13:17.468123 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.468006    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fk7ml" event={"ID":"f4ef6497-03ed-4ed8-beb9-1cde1f435655","Type":"ContainerStarted","Data":"fc863551f4e03661adb4528ce907d4054b5bdf6c42cafa5037cdb3ddf42a6ea4"}
Oct 29 23:13:17.470206 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.470141    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dn5kk" event={"ID":"59d798e8-802a-4a6a-8f5a-063d69972c13","Type":"ContainerStarted","Data":"76edbc97b64d77cc4eb3bd88ad6e43e5ab4f389627af1d1d293c28c93aeb7d4b"}
Oct 29 23:13:17.471892 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.471839    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mnl8f" event={"ID":"802fe00a-6d8f-483e-9fb5-2ea266688ad1","Type":"ContainerStarted","Data":"6b450ddcc054ff72ae223cb160cfaf267a5093104ac64e78998c8ac0553345f3"}
Oct 29 23:13:17.850992 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.849339    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 29 23:13:17.854167 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.853657    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.877665 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.875761    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 29 23:13:17.877665 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.875862    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 29 23:13:17.877665 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.877182    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 29 23:13:17.877665 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.877263    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 29 23:13:17.877665 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.877327    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 29 23:13:17.877665 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.877426    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 29 23:13:17.877665 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.877491    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fnb2j\""
Oct 29 23:13:17.877665 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.877531    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 29 23:13:17.877665 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.877565    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 29 23:13:17.877665 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.877614    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 29 23:13:17.893188 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.893076    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 29 23:13:17.954950 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.954912    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-web-config\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955127 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.954984    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955127 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.955029    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955127 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.955056    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955127 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.955085    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-volume\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955127 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.955111    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwpxg\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-kube-api-access-pwpxg\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.955157    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.955184    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.955226    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.955260    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.955285    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.955313    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:17.955370 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:17.955343    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-out\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056430    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-out\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056489    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-web-config\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056542    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056590    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056620    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056649    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-volume\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056680    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwpxg\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-kube-api-access-pwpxg\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056730    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056772    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056815    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056851    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056871    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.057647 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.056894    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.059452 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.058870    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.060170 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.060144    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-volume\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.061187 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.061160    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.061858 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.061784    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.062618 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.062593    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.066591 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.066564    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-web-config\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.068305 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.068276    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-out\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.070136 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.070090    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.070765 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.070711    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.071125 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.070960    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.071125 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.071087    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.073122 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.073099    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.089389 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.089359    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwpxg\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-kube-api-access-pwpxg\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.183674 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.183643    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:18.476230 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.476190    2574 generic.go:356] "Generic (PLEG): container finished" podID="59d798e8-802a-4a6a-8f5a-063d69972c13" containerID="52ec78d673ec09bc028c431ef2c529e76f1901bca8d98b290fb1900cc0021a00" exitCode=0
Oct 29 23:13:18.476672 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.476263    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dn5kk" event={"ID":"59d798e8-802a-4a6a-8f5a-063d69972c13","Type":"ContainerDied","Data":"52ec78d673ec09bc028c431ef2c529e76f1901bca8d98b290fb1900cc0021a00"}
Oct 29 23:13:18.543118 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:18.543020    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 29 23:13:18.546792 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:13:18.546764    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf091e44_0c38_428f_a14f_f3a44f9b178b.slice/crio-bbcdc18091b22a47939271a133581135955337afb04756ce9d6709a4521e29b8 WatchSource:0}: Error finding container bbcdc18091b22a47939271a133581135955337afb04756ce9d6709a4521e29b8: Status 404 returned error can't find the container with id bbcdc18091b22a47939271a133581135955337afb04756ce9d6709a4521e29b8
Oct 29 23:13:19.486867 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:19.486758    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fk7ml" event={"ID":"f4ef6497-03ed-4ed8-beb9-1cde1f435655","Type":"ContainerStarted","Data":"72fd67e91a526500a88054aaf54d3d7379b650bd8478ebef5df3d9bc50211172"}
Oct 29 23:13:19.488262 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:19.488236    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerStarted","Data":"bbcdc18091b22a47939271a133581135955337afb04756ce9d6709a4521e29b8"}
Oct 29 23:13:19.490437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:19.490409    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dn5kk" event={"ID":"59d798e8-802a-4a6a-8f5a-063d69972c13","Type":"ContainerStarted","Data":"41d16b69ef65ec5769850995df8b3cd1cdfa4d011275261d03f7a5420906b4de"}
Oct 29 23:13:19.490533 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:19.490460    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dn5kk" event={"ID":"59d798e8-802a-4a6a-8f5a-063d69972c13","Type":"ContainerStarted","Data":"3edc646b1dbc1df1bd5a2d496508abfe9ccb1eb8a6682208095e182a6fca3b96"}
Oct 29 23:13:19.492195 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:19.492175    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mnl8f" event={"ID":"802fe00a-6d8f-483e-9fb5-2ea266688ad1","Type":"ContainerStarted","Data":"533833b632bca542b4898db59af7e3eea2804e1b0484bad0bd51b21bc42a0d8c"}
Oct 29 23:13:19.492281 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:19.492216    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mnl8f" event={"ID":"802fe00a-6d8f-483e-9fb5-2ea266688ad1","Type":"ContainerStarted","Data":"04c75ab87fc595372d153047aa8b8495562d20dc82c9bb649a79329f045cd7ad"}
Oct 29 23:13:19.492346 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:19.492330    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:19.531437 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:19.531385    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fk7ml" podStartSLOduration=2.212995856 podStartE2EDuration="4.531368854s" totalImagesPullingTime="2.318372998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:13:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:19.531100027 +0000 UTC m=+51.870557707" watchObservedRunningTime="2026-10-29 23:13:19.531368854 +0000 UTC m=+51.870826534"
Oct 29 23:13:19.579422 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:19.579370    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dn5kk" podStartSLOduration=2.281094865 podStartE2EDuration="3.579355818s" totalImagesPullingTime="1.199304104s" totalInitContainerRuntime="98.956849ms" isStatefulPod=true podCreationTimestamp="2026-10-29 23:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:19.578123493 +0000 UTC m=+51.917581175" watchObservedRunningTime="2026-10-29 23:13:19.579355818 +0000 UTC m=+51.918813498"
Oct 29 23:13:19.625128 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:19.625084    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-mnl8f" podStartSLOduration=2.649033062 podStartE2EDuration="4.625069915s" totalImagesPullingTime="1.976036853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:19.62465535 +0000 UTC m=+51.964113030" watchObservedRunningTime="2026-10-29 23:13:19.625069915 +0000 UTC m=+51.964527591"
Oct 29 23:13:20.496567 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:20.496529    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qlpqj" event={"ID":"9eb8b8c3-aae3-4116-9d9e-244d34ba0271","Type":"ContainerStarted","Data":"6a6babda17c4ea715da6254119e82db15a7ccd0c948a0f3e276261b85814d38a"}
Oct 29 23:13:20.498140 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:20.498115    2574 generic.go:356] "Generic (PLEG): container finished" podID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerID="0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66" exitCode=0
Oct 29 23:13:20.498232 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:20.498211    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerDied","Data":"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"}
Oct 29 23:13:20.537783 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:20.537723    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qlpqj" podStartSLOduration=2.5261725029999997 podStartE2EDuration="5.537710455s" totalImagesPullingTime="3.011537952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:20.536532078 +0000 UTC m=+52.875989759" watchObservedRunningTime="2026-10-29 23:13:20.537710455 +0000 UTC m=+52.877168134"
Oct 29 23:13:21.103445 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.103408    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"]
Oct 29 23:13:21.106676 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.106654    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.125438 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.125413    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4d7v87a6le8jl\""
Oct 29 23:13:21.137879 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.137851    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 29 23:13:21.137997 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.137887    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 29 23:13:21.137997 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.137969    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pk5sf\""
Oct 29 23:13:21.138107 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.138042    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 29 23:13:21.143545 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.143515    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"]
Oct 29 23:13:21.229364 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.229331    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 29 23:13:21.288014 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.287974    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cbf0e826-b477-47a2-9c6d-08ac105aedd0-audit-log\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.288187 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.288056    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-client-certs\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.288187 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.288109    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.288187 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.288133    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-tls\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.288356 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.288192    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-metrics-server-audit-profiles\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.288356 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.288302    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-client-ca-bundle\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.288434 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.288383    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qf9c\" (UniqueName: \"kubernetes.io/projected/cbf0e826-b477-47a2-9c6d-08ac105aedd0-kube-api-access-2qf9c\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.389231 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.389199    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cbf0e826-b477-47a2-9c6d-08ac105aedd0-audit-log\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.389422 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.389257    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-client-certs\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.389422 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.389302    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.389422 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.389330    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-tls\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.389422 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.389355    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-metrics-server-audit-profiles\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.389422 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.389383    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-client-ca-bundle\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.389643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.389439    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qf9c\" (UniqueName: \"kubernetes.io/projected/cbf0e826-b477-47a2-9c6d-08ac105aedd0-kube-api-access-2qf9c\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.389707 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.389674    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cbf0e826-b477-47a2-9c6d-08ac105aedd0-audit-log\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.390484 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.390444    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-metrics-server-audit-profiles\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.390626 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.390519    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.392262 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.392173    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-client-certs\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.392373 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.392309    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-client-ca-bundle\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.392943 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.392922    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-tls\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.412984 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.412955    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qf9c\" (UniqueName: \"kubernetes.io/projected/cbf0e826-b477-47a2-9c6d-08ac105aedd0-kube-api-access-2qf9c\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.421038 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.420814    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:21.739643 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.739622    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"]
Oct 29 23:13:21.741928 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:13:21.741896    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbf0e826_b477_47a2_9c6d_08ac105aedd0.slice/crio-b1499e2c5e1aa06e2198d073e65da4ab6655f5900a0e182f9d725660a5342897 WatchSource:0}: Error finding container b1499e2c5e1aa06e2198d073e65da4ab6655f5900a0e182f9d725660a5342897: Status 404 returned error can't find the container with id b1499e2c5e1aa06e2198d073e65da4ab6655f5900a0e182f9d725660a5342897
Oct 29 23:13:21.839681 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.839648    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-79699f4d76-w6hb5"]
Oct 29 23:13:21.842879 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.842857    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.859067 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.859046    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 29 23:13:21.859168 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.859080    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 29 23:13:21.859168 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.859103    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 29 23:13:21.859168 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.859124    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 29 23:13:21.859362 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.859349    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 29 23:13:21.859448 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.859436    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lpzjf\""
Oct 29 23:13:21.884912 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.884885    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 29 23:13:21.886062 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.886042    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-79699f4d76-w6hb5"]
Oct 29 23:13:21.892848 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.892820    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.892931 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.892886    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-serving-certs-ca-bundle\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.893017 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.892997    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdgp9\" (UniqueName: \"kubernetes.io/projected/259771b7-e14a-496a-bba1-a8f7364e6c1f-kube-api-access-gdgp9\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.893223 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.893201    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-federate-client-tls\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.893288 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.893252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-client-tls\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.893288 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.893272    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-metrics-client-ca\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.893388 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.893290    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.893388 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.893341    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.993798 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.993774    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdgp9\" (UniqueName: \"kubernetes.io/projected/259771b7-e14a-496a-bba1-a8f7364e6c1f-kube-api-access-gdgp9\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.993887 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.993810    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-federate-client-tls\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.993887 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.993838    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-client-tls\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.993887 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.993856    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-metrics-client-ca\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.993887 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.993883    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.994049 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.993902    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.994049 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.993920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.994049 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.993973    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-serving-certs-ca-bundle\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.994781 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.994722    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-serving-certs-ca-bundle\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.994899 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.994834    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-metrics-client-ca\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.994965 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.994922    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.996644 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.996617    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-federate-client-tls\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.996755 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.996627    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.996807 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.996784    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:21.996841 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:21.996787    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-client-tls\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:22.017495 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:22.017468    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdgp9\" (UniqueName: \"kubernetes.io/projected/259771b7-e14a-496a-bba1-a8f7364e6c1f-kube-api-access-gdgp9\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:22.158843 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:22.158635    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:22.504724 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:22.504682    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-79699f4d76-w6hb5"]
Oct 29 23:13:22.510327 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:22.508303    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw" event={"ID":"cbf0e826-b477-47a2-9c6d-08ac105aedd0","Type":"ContainerStarted","Data":"b1499e2c5e1aa06e2198d073e65da4ab6655f5900a0e182f9d725660a5342897"}
Oct 29 23:13:22.513046 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:22.513016    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerStarted","Data":"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"}
Oct 29 23:13:22.513178 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:22.513087    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerStarted","Data":"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"}
Oct 29 23:13:22.513178 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:22.513127    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerStarted","Data":"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"}
Oct 29 23:13:22.513178 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:22.513160    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerStarted","Data":"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"}
Oct 29 23:13:22.513330 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:22.513195    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerStarted","Data":"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"}
Oct 29 23:13:22.517989 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:13:22.517963    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod259771b7_e14a_496a_bba1_a8f7364e6c1f.slice/crio-933e890befee0f4c409fe216775374657a533bb4257324c695ea765223e96eb6 WatchSource:0}: Error finding container 933e890befee0f4c409fe216775374657a533bb4257324c695ea765223e96eb6: Status 404 returned error can't find the container with id 933e890befee0f4c409fe216775374657a533bb4257324c695ea765223e96eb6
Oct 29 23:13:23.419153 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:23.419122    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:23.518082 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:23.518045    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5" event={"ID":"259771b7-e14a-496a-bba1-a8f7364e6c1f","Type":"ContainerStarted","Data":"933e890befee0f4c409fe216775374657a533bb4257324c695ea765223e96eb6"}
Oct 29 23:13:23.519954 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:23.519912    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw" event={"ID":"cbf0e826-b477-47a2-9c6d-08ac105aedd0","Type":"ContainerStarted","Data":"9fdbceb55dc4e5d2b41de91f51d753d1b89bce17e26229d8794d58de6b04565e"}
Oct 29 23:13:23.523859 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:23.523836    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerStarted","Data":"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"}
Oct 29 23:13:23.563548 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:23.563499    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw" podStartSLOduration=1.290505215 podStartE2EDuration="2.563488042s" totalImagesPullingTime="1.272982827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:23.562513664 +0000 UTC m=+55.901971344" watchObservedRunningTime="2026-10-29 23:13:23.563488042 +0000 UTC m=+55.902945721"
Oct 29 23:13:23.646183 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:23.646125    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.496077891 podStartE2EDuration="6.646109332s" totalImagesPullingTime="3.136776757s" totalInitContainerRuntime="13.254684ms" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:23.644388828 +0000 UTC m=+55.983846508" watchObservedRunningTime="2026-10-29 23:13:23.646109332 +0000 UTC m=+55.985567011"
Oct 29 23:13:24.532837 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:24.532728    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5" event={"ID":"259771b7-e14a-496a-bba1-a8f7364e6c1f","Type":"ContainerStarted","Data":"18a1983c155c88b96620f003be8ed6860b45936178df93246d5e3faf790f70db"}
Oct 29 23:13:24.533247 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:24.532856    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5" event={"ID":"259771b7-e14a-496a-bba1-a8f7364e6c1f","Type":"ContainerStarted","Data":"da0a2b8a6ee6884e3d222c5639443b559aa82dfe4f1e66c984e5d78394436f46"}
Oct 29 23:13:24.533247 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:24.532880    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5" event={"ID":"259771b7-e14a-496a-bba1-a8f7364e6c1f","Type":"ContainerStarted","Data":"8635b6789e54016891c1462af412752d4e8d15748261f14b52ef4dde2fd44362"}
Oct 29 23:13:24.603229 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:24.603151    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5" podStartSLOduration=1.893674222 podStartE2EDuration="3.603137664s" totalImagesPullingTime="1.709463442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:24.602175519 +0000 UTC m=+56.941633198" watchObservedRunningTime="2026-10-29 23:13:24.603137664 +0000 UTC m=+56.942595343"
Oct 29 23:13:29.505543 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:29.505515    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:30.646933 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.646899    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-9jhw5"]
Oct 29 23:13:30.651215 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.651197    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:30.682329 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.682303    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-9jhw5"]
Oct 29 23:13:30.692873 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.692851    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 29 23:13:30.692873 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.692873    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 29 23:13:30.751077 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.751047    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-cwxm6\""
Oct 29 23:13:30.768386 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.768356    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnw2k\" (UniqueName: \"kubernetes.io/projected/ed669269-4708-416f-8b97-9ee2cc5bf891-kube-api-access-xnw2k\") pod \"downloads-6dd789c8b7-9jhw5\" (UID: \"ed669269-4708-416f-8b97-9ee2cc5bf891\") " pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:30.768519 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.768447    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed669269-4708-416f-8b97-9ee2cc5bf891-tmp\") pod \"downloads-6dd789c8b7-9jhw5\" (UID: \"ed669269-4708-416f-8b97-9ee2cc5bf891\") " pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:30.869528 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.869498    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed669269-4708-416f-8b97-9ee2cc5bf891-tmp\") pod \"downloads-6dd789c8b7-9jhw5\" (UID: \"ed669269-4708-416f-8b97-9ee2cc5bf891\") " pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:30.869696 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.869557    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnw2k\" (UniqueName: \"kubernetes.io/projected/ed669269-4708-416f-8b97-9ee2cc5bf891-kube-api-access-xnw2k\") pod \"downloads-6dd789c8b7-9jhw5\" (UID: \"ed669269-4708-416f-8b97-9ee2cc5bf891\") " pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:30.869873 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.869852    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed669269-4708-416f-8b97-9ee2cc5bf891-tmp\") pod \"downloads-6dd789c8b7-9jhw5\" (UID: \"ed669269-4708-416f-8b97-9ee2cc5bf891\") " pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:30.910566 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.910505    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnw2k\" (UniqueName: \"kubernetes.io/projected/ed669269-4708-416f-8b97-9ee2cc5bf891-kube-api-access-xnw2k\") pod \"downloads-6dd789c8b7-9jhw5\" (UID: \"ed669269-4708-416f-8b97-9ee2cc5bf891\") " pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:30.961930 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:30.961894    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:31.287575 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:31.287496    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-9jhw5"]
Oct 29 23:13:31.301113 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:13:31.301084    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded669269_4708_416f_8b97_9ee2cc5bf891.slice/crio-7345dd53d558203baf30a745aa105d73e0a53547a2aaf7043638d52b28f30bae WatchSource:0}: Error finding container 7345dd53d558203baf30a745aa105d73e0a53547a2aaf7043638d52b28f30bae: Status 404 returned error can't find the container with id 7345dd53d558203baf30a745aa105d73e0a53547a2aaf7043638d52b28f30bae
Oct 29 23:13:31.556197 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:31.556119    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-9jhw5" event={"ID":"ed669269-4708-416f-8b97-9ee2cc5bf891","Type":"ContainerStarted","Data":"7345dd53d558203baf30a745aa105d73e0a53547a2aaf7043638d52b28f30bae"}
Oct 29 23:13:33.997315 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:33.997224    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:33.997791 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:33.997322    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:34.011794 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.011703    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 29 23:13:34.020111 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.020080    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:34.023929 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.023899    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 29 23:13:34.030426 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.030397    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:34.098549 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.098514    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:34.113501 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.113468    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 29 23:13:34.133992 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.133957    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 29 23:13:34.142806 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.142777    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:34.174955 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.174918    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wscf4\""
Oct 29 23:13:34.179122 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.179101    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-m6t5c\""
Oct 29 23:13:34.181205 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.181182    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:34.183073 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.183051    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-z28mk\""
Oct 29 23:13:34.185256 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.185240    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:34.189147 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.189123    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:34.503890 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.503864    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bv5tv"]
Oct 29 23:13:34.505222 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:13:34.505194    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ec6811_5291_4963_b8f1_52a7ffdc50bc.slice/crio-2c18767b52ddca68d5aea85ec40d8c544d926d002bffd0b251c586cb6d2b5311 WatchSource:0}: Error finding container 2c18767b52ddca68d5aea85ec40d8c544d926d002bffd0b251c586cb6d2b5311: Status 404 returned error can't find the container with id 2c18767b52ddca68d5aea85ec40d8c544d926d002bffd0b251c586cb6d2b5311
Oct 29 23:13:34.521292 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.521270    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wht7s"]
Oct 29 23:13:34.522141 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:13:34.522118    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ac2ba2_0d4b_4498_a7c6_658412d60ccf.slice/crio-d5b0afdb5cf53716a257e3d3008322004dde10191eb4e0005e53f73199063f3f WatchSource:0}: Error finding container d5b0afdb5cf53716a257e3d3008322004dde10191eb4e0005e53f73199063f3f: Status 404 returned error can't find the container with id d5b0afdb5cf53716a257e3d3008322004dde10191eb4e0005e53f73199063f3f
Oct 29 23:13:34.528289 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.528263    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xhlf6"]
Oct 29 23:13:34.529035 ip-10-0-138-189 kubenswrapper[2574]: W1029 23:13:34.529004    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1c395e_bc72_4038_80cc_4ab131d96fdb.slice/crio-6ee0426bf15653002e95687e82fc07c34e8202073a2f5cc46e68dda74aefd9e1 WatchSource:0}: Error finding container 6ee0426bf15653002e95687e82fc07c34e8202073a2f5cc46e68dda74aefd9e1: Status 404 returned error can't find the container with id 6ee0426bf15653002e95687e82fc07c34e8202073a2f5cc46e68dda74aefd9e1
Oct 29 23:13:34.568415 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.568377    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xhlf6" event={"ID":"fd1c395e-bc72-4038-80cc-4ab131d96fdb","Type":"ContainerStarted","Data":"6ee0426bf15653002e95687e82fc07c34e8202073a2f5cc46e68dda74aefd9e1"}
Oct 29 23:13:34.569797 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.569770    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bv5tv" event={"ID":"62ec6811-5291-4963-b8f1-52a7ffdc50bc","Type":"ContainerStarted","Data":"2c18767b52ddca68d5aea85ec40d8c544d926d002bffd0b251c586cb6d2b5311"}
Oct 29 23:13:34.571030 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:34.570996    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wht7s" event={"ID":"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf","Type":"ContainerStarted","Data":"d5b0afdb5cf53716a257e3d3008322004dde10191eb4e0005e53f73199063f3f"}
Oct 29 23:13:36.584835 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:36.584788    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bv5tv" event={"ID":"62ec6811-5291-4963-b8f1-52a7ffdc50bc","Type":"ContainerStarted","Data":"3d6b168eb657ec9f1ca8461453aee545c14babdc098849b9a255acaf738c902b"}
Oct 29 23:13:36.585314 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:36.584859    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bv5tv" event={"ID":"62ec6811-5291-4963-b8f1-52a7ffdc50bc","Type":"ContainerStarted","Data":"7bc3ea141f9c9331129170bad0499b019163d8bdf781f64f4dcdc027c3c2e14e"}
Oct 29 23:13:36.634768 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:36.634694    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bv5tv" podStartSLOduration=66.461989475 podStartE2EDuration="1m7.634674707s" totalImagesPullingTime="1.172685232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:36.634213759 +0000 UTC m=+68.973671440" watchObservedRunningTime="2026-10-29 23:13:36.634674707 +0000 UTC m=+68.974132388"
Oct 29 23:13:38.594695 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:38.594604    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xhlf6" event={"ID":"fd1c395e-bc72-4038-80cc-4ab131d96fdb","Type":"ContainerStarted","Data":"4c2cddf3d669f79b7ea6cd8ef5d00ae82bc86b4afdd63eedb13dc91ba6f0e4a9"}
Oct 29 23:13:38.595192 ip-10-0-138-189 kubenswrapper[2574]: I1029 23:13:38.594763    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:40.231495 ip-10-0-138-189 systemd[1]: Stopping Kubernetes Kubelet...
Oct 29 23:13:40.237859 ip-10-0-138-189 systemd[1]: kubelet.service: Deactivated successfully.
Oct 29 23:13:40.238049 ip-10-0-138-189 systemd[1]: Stopped Kubernetes Kubelet.
Oct 29 23:13:40.238104 ip-10-0-138-189 systemd[1]: kubelet.service: Consumed 3.908s CPU time.
Oct 29 23:13:40.252340 ip-10-0-138-189 systemd[1]: Starting Kubernetes Kubelet...
Oct 29 23:13:40.335907 ip-10-0-138-189 kubenswrapper[6040]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:13:40.335907 ip-10-0-138-189 kubenswrapper[6040]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 29 23:13:40.335907 ip-10-0-138-189 kubenswrapper[6040]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:13:40.335907 ip-10-0-138-189 kubenswrapper[6040]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:13:40.338057 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.337968    6040 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:40.338057 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338052    6040 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:40.338057 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338057    6040 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:40.338057 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338060    6040 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:40.338057 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338063    6040 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:40.338057 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338066    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338069    6040 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338072    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338075    6040 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338077    6040 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338080    6040 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338083    6040 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338087    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338090    6040 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338092    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338095    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338098    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338100    6040 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338103    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338106    6040 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338108    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338111    6040 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338114    6040 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338116    6040 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:40.338327 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338119    6040 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338122    6040 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338124    6040 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338127    6040 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338134    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338137    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338140    6040 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338142    6040 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338145    6040 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338148    6040 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338150    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338153    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338156    6040 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338159    6040 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338162    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338165    6040 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338168    6040 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338170    6040 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338173    6040 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338176    6040 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:40.338816 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338178    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338181    6040 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338183    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338186    6040 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338189    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338192    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338195    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338197    6040 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338200    6040 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338202    6040 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338205    6040 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338208    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338210    6040 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338213    6040 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338216    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338219    6040 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338222    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338224    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338227    6040 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:40.339364 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338230    6040 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338233    6040 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338235    6040 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338238    6040 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338241    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338243    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338246    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338249    6040 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338251    6040 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338254    6040 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338256    6040 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338260    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338263    6040 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338265    6040 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338268    6040 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338271    6040 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338275    6040 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338279    6040 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338282    6040 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338285    6040 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:40.339843 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338288    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338291    6040 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338294    6040 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338297    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338299    6040 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338302    6040 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338304    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338307    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338310    6040 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338312    6040 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338315    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338317    6040 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338320    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338323    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338326    6040 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338331    6040 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338334    6040 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338337    6040 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338340    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338343    6040 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:40.340321 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338345    6040 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338351    6040 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.338354    6040 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339126    6040 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339134    6040 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339137    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339140    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339143    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339146    6040 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339149    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339152    6040 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339155    6040 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339158    6040 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339161    6040 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339164    6040 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339167    6040 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339170    6040 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339172    6040 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339175    6040 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339178    6040 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:40.340817 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339181    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339184    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339187    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339189    6040 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339192    6040 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339195    6040 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339197    6040 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339202    6040 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339205    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339208    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339211    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339213    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339216    6040 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339218    6040 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339221    6040 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339224    6040 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339226    6040 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339229    6040 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339231    6040 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:40.341306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339234    6040 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339237    6040 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339240    6040 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339242    6040 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339245    6040 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339247    6040 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339250    6040 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339253    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339255    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339258    6040 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339261    6040 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339264    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339267    6040 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339269    6040 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339272    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339275    6040 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339277    6040 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339280    6040 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339283    6040 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339285    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:40.341789 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339290    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339292    6040 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339295    6040 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339297    6040 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339300    6040 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339302    6040 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339305    6040 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339307    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339310    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339313    6040 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339315    6040 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339318    6040 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339322    6040 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339326    6040 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339329    6040 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339333    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339335    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339338    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339341    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:40.342293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339344    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339347    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339349    6040 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339352    6040 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339354    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339357    6040 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339360    6040 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339362    6040 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339365    6040 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339368    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339370    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339373    6040 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339376    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339380    6040 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339382    6040 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339385    6040 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339388    6040 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339390    6040 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339393    6040 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:40.342768 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339395    6040 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339398    6040 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339401    6040 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339403    6040 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339406    6040 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339408    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339411    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339413    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339416    6040 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339420    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339425    6040 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339429    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339907    6040 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339913    6040 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339917    6040 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339920    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339923    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339926    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339929    6040 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:40.343247 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339932    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339935    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339938    6040 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339941    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339943    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339946    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339949    6040 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339952    6040 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339955    6040 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339957    6040 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339960    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339963    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339967    6040 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339970    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339973    6040 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339975    6040 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339978    6040 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339980    6040 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339983    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339985    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:40.343787 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339988    6040 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339990    6040 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.339994    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340001    6040 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340004    6040 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340006    6040 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340009    6040 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340012    6040 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340015    6040 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340017    6040 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340020    6040 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340022    6040 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340024    6040 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340027    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340030    6040 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340032    6040 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340035    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340037    6040 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340040    6040 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340043    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:40.344259 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340046    6040 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340048    6040 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340051    6040 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340053    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340056    6040 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340059    6040 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340061    6040 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340064    6040 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340067    6040 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340070    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340072    6040 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340075    6040 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340077    6040 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340080    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340082    6040 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340085    6040 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340089    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340092    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340095    6040 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340097    6040 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:40.344806 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340100    6040 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340102    6040 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340105    6040 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340107    6040 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340110    6040 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340112    6040 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340115    6040 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340118    6040 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340120    6040 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340123    6040 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340125    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340128    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340132    6040 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340136    6040 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340138    6040 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340141    6040 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340145    6040 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340148    6040 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340152    6040 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:40.345293 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340154    6040 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340157    6040 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340160    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340162    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340165    6040 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340167    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340170    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340172    6040 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340175    6040 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340177    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340180    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340183    6040 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340186    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340188    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340191    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340194    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340196    6040 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340199    6040 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340202    6040 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340204    6040 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:40.345778 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340293    6040 flags.go:64] FLAG: --address="0.0.0.0"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340300    6040 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340307    6040 flags.go:64] FLAG: --anonymous-auth="true"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340311    6040 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340318    6040 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340323    6040 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340333    6040 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340337    6040 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340340    6040 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340343    6040 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340347    6040 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340351    6040 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340354    6040 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340357    6040 flags.go:64] FLAG: --cgroup-root=""
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340360    6040 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340364    6040 flags.go:64] FLAG: --client-ca-file=""
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340367    6040 flags.go:64] FLAG: --cloud-provider="external"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340370    6040 flags.go:64] FLAG: --cluster-dns="[]"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340374    6040 flags.go:64] FLAG: --cluster-domain=""
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340377    6040 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340380    6040 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340383    6040 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340387    6040 flags.go:64] FLAG: --container-log-max-files="5"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340392    6040 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 29 23:13:40.346296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340396    6040 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340399    6040 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340402    6040 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340405    6040 flags.go:64] FLAG: --contention-profiling="false"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340408    6040 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340411    6040 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340414    6040 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340417    6040 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340423    6040 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340426    6040 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340431    6040 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340434    6040 flags.go:64] FLAG: --enable-load-reader="false"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340437    6040 flags.go:64] FLAG: --enable-server="true"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340440    6040 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340445    6040 flags.go:64] FLAG: --event-burst="100"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340449    6040 flags.go:64] FLAG: --event-qps="50"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340452    6040 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340455    6040 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340458    6040 flags.go:64] FLAG: --eviction-hard=""
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340462    6040 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340466    6040 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340469    6040 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340472    6040 flags.go:64] FLAG: --eviction-soft=""
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340475    6040 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340478    6040 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 29 23:13:40.346923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340481    6040 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340484    6040 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340488    6040 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340491    6040 flags.go:64] FLAG: --fail-swap-on="true"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340494    6040 flags.go:64] FLAG: --feature-gates=""
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340498    6040 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340501    6040 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340505    6040 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340508    6040 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340511    6040 flags.go:64] FLAG: --healthz-port="10248"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340515    6040 flags.go:64] FLAG: --help="false"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340518    6040 flags.go:64] FLAG: --hostname-override="ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340521    6040 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340524    6040 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340528    6040 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340534    6040 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340538    6040 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340542    6040 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340545    6040 flags.go:64] FLAG: --image-service-endpoint=""
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340548    6040 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340551    6040 flags.go:64] FLAG: --kube-api-burst="100"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340554    6040 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340558    6040 flags.go:64] FLAG: --kube-api-qps="50"
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340561    6040 flags.go:64] FLAG: --kube-reserved=""
Oct 29 23:13:40.347534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340565    6040 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340567    6040 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340570    6040 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340573    6040 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340576    6040 flags.go:64] FLAG: --lock-file=""
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340579    6040 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340583    6040 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340585    6040 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340591    6040 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340594    6040 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340597    6040 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340600    6040 flags.go:64] FLAG: --logging-format="text"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340603    6040 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340607    6040 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340610    6040 flags.go:64] FLAG: --manifest-url=""
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340613    6040 flags.go:64] FLAG: --manifest-url-header=""
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340618    6040 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340621    6040 flags.go:64] FLAG: --max-open-files="1000000"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340625    6040 flags.go:64] FLAG: --max-pods="110"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340628    6040 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340632    6040 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340635    6040 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340638    6040 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340642    6040 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340645    6040 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 29 23:13:40.348126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340649    6040 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340656    6040 flags.go:64] FLAG: --node-status-max-images="50"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340660    6040 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340663    6040 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340666    6040 flags.go:64] FLAG: --pod-cidr=""
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340669    6040 flags.go:64] FLAG: --pod-manifest-path=""
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340672    6040 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340676    6040 flags.go:64] FLAG: --pods-per-core="0"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340679    6040 flags.go:64] FLAG: --port="10250"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340682    6040 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340685    6040 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07cee1894d1cf65f8"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340688    6040 flags.go:64] FLAG: --qos-reserved=""
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340691    6040 flags.go:64] FLAG: --read-only-port="10255"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340694    6040 flags.go:64] FLAG: --register-node="true"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340697    6040 flags.go:64] FLAG: --register-with-taints=""
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340701    6040 flags.go:64] FLAG: --registry-burst="10"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340704    6040 flags.go:64] FLAG: --registry-qps="5"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340707    6040 flags.go:64] FLAG: --reserved-cpus=""
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340710    6040 flags.go:64] FLAG: --reserved-memory=""
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340714    6040 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340717    6040 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340721    6040 flags.go:64] FLAG: --rotate-certificates="false"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340724    6040 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340727    6040 flags.go:64] FLAG: --runonce="false"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340730    6040 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340733    6040 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340750    6040 flags.go:64] FLAG: --seccomp-default="false"
Oct 29 23:13:40.348763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340753    6040 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340756    6040 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340759    6040 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340764    6040 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340767    6040 flags.go:64] FLAG: --storage-driver-password="root"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340771    6040 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340774    6040 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340777    6040 flags.go:64] FLAG: --storage-driver-user="root"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340780    6040 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340783    6040 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340787    6040 flags.go:64] FLAG: --system-cgroups=""
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340790    6040 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340795    6040 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340798    6040 flags.go:64] FLAG: --tls-cert-file=""
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340801    6040 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340805    6040 flags.go:64] FLAG: --tls-min-version=""
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340808    6040 flags.go:64] FLAG: --tls-private-key-file=""
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340811    6040 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340814    6040 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340817    6040 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340820    6040 flags.go:64] FLAG: --v="2"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340824    6040 flags.go:64] FLAG: --version="false"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340829    6040 flags.go:64] FLAG: --vmodule=""
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340833    6040 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.340836    6040 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 29 23:13:40.349594 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340973    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340977    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340980    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340983    6040 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340986    6040 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340992    6040 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340995    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.340999    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341001    6040 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341004    6040 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341008    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341011    6040 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341017    6040 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341021    6040 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341024    6040 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341027    6040 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341030    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341033    6040 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341035    6040 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:40.350509 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341038    6040 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341041    6040 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341045    6040 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341049    6040 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341051    6040 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341054    6040 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341057    6040 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341059    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341062    6040 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341064    6040 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341067    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341069    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341072    6040 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341075    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341077    6040 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341080    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341082    6040 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341085    6040 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341088    6040 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341091    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:40.351016 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341094    6040 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341097    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341100    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341104    6040 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341106    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341110    6040 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341113    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341116    6040 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341119    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341122    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341124    6040 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341127    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341129    6040 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341132    6040 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341135    6040 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341138    6040 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341140    6040 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341143    6040 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341146    6040 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:40.351543 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341148    6040 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341151    6040 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341154    6040 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341157    6040 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341159    6040 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341162    6040 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341165    6040 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341167    6040 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341170    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341173    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341175    6040 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341178    6040 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341180    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341183    6040 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341186    6040 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341189    6040 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341193    6040 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341195    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341199    6040 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341202    6040 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:40.352177 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341205    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341207    6040 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341210    6040 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341213    6040 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341215    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341218    6040 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341221    6040 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341223    6040 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341226    6040 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341228    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341232    6040 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341234    6040 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341237    6040 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341239    6040 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341242    6040 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341245    6040 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341247    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341250    6040 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341253    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341255    6040 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:40.352715 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341258    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341260    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341263    6040 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341266    6040 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341268    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341271    6040 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341274    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341277    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.341304    6040 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341345    6040 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341351    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341354    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341357    6040 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341360    6040 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341362    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341365    6040 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:40.353224 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341368    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341371    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341373    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341376    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341379    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341381    6040 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341384    6040 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341386    6040 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341389    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341392    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341395    6040 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341397    6040 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341400    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341403    6040 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341407    6040 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341411    6040 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341414    6040 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341416    6040 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341419    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:40.353619 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341421    6040 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341424    6040 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341427    6040 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341430    6040 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341435    6040 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341439    6040 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341444    6040 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341449    6040 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341452    6040 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341454    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341457    6040 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341460    6040 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341462    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341465    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341468    6040 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341470    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341473    6040 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341475    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341478    6040 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341481    6040 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:40.354137 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341483    6040 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341486    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341490    6040 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341492    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341495    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341497    6040 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341500    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341503    6040 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341506    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341508    6040 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341511    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341514    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341516    6040 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341519    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341521    6040 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341524    6040 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341527    6040 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341530    6040 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341534    6040 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:40.354792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341538    6040 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341541    6040 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341543    6040 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341546    6040 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341549    6040 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341552    6040 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341554    6040 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341557    6040 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341560    6040 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341562    6040 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341565    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341567    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341570    6040 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341572    6040 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341575    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341577    6040 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341580    6040 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341583    6040 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341585    6040 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341588    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:40.355306 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341590    6040 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341593    6040 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341596    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341598    6040 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341601    6040 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341604    6040 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341606    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341609    6040 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341611    6040 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341614    6040 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341616    6040 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341621    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341624    6040 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341627    6040 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341630    6040 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341632    6040 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341635    6040 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341638    6040 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341641    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341644    6040 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:40.355884 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.341646    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.345229    6040 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.345246    6040 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.345264    6040 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.345269    6040 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345326    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345332    6040 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345335    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345338    6040 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345341    6040 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345344    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345347    6040 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345350    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345353    6040 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345355    6040 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345358    6040 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345361    6040 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345364    6040 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345367    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345370    6040 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:40.356355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345373    6040 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345375    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345378    6040 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345382    6040 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345385    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345387    6040 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345390    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345393    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345395    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345398    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345401    6040 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345404    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345407    6040 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345410    6040 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345413    6040 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345415    6040 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345418    6040 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345421    6040 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345424    6040 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345427    6040 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:40.356853 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345430    6040 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345433    6040 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345436    6040 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345439    6040 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345442    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345444    6040 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345447    6040 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345450    6040 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345453    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345456    6040 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345459    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345461    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345464    6040 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345467    6040 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345469    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345472    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345475    6040 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345478    6040 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345481    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:40.357336 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345483    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345486    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345488    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345491    6040 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345494    6040 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345497    6040 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345500    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345502    6040 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345505    6040 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345508    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345511    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345514    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345517    6040 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345519    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345522    6040 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345526    6040 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345530    6040 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345533    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345536    6040 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:40.357860 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345539    6040 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345542    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345544    6040 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345547    6040 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345550    6040 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345554    6040 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345558    6040 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345561    6040 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345564    6040 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345567    6040 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345570    6040 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345573    6040 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345576    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345578    6040 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345581    6040 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345584    6040 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345587    6040 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345589    6040 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345592    6040 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:40.358305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345595    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345598    6040 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345600    6040 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345603    6040 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345605    6040 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345608    6040 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345611    6040 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345614    6040 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345616    6040 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345619    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345622    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345624    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345627    6040 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345630    6040 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.345656    6040 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345849    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:40.358788 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345856    6040 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345860    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345863    6040 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345866    6040 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345869    6040 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345872    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345875    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345878    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345882    6040 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345887    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345890    6040 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345893    6040 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345896    6040 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345898    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345901    6040 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345904    6040 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345907    6040 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345909    6040 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345912    6040 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:40.359208 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345914    6040 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345917    6040 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345921    6040 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345925    6040 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345929    6040 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345933    6040 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345937    6040 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345941    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345945    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345949    6040 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345952    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345954    6040 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345957    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345960    6040 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345962    6040 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345965    6040 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345967    6040 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345970    6040 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345973    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:40.359658 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345975    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345978    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345981    6040 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345984    6040 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345987    6040 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345991    6040 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345995    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.345998    6040 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346002    6040 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346005    6040 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346009    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346013    6040 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346018    6040 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346022    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346026    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346029    6040 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346032    6040 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346035    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346038    6040 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:40.360241 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346040    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346043    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346046    6040 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346048    6040 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346051    6040 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346054    6040 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346056    6040 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346059    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346061    6040 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346064    6040 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346067    6040 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346069    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346072    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346074    6040 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346077    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346080    6040 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346083    6040 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346086    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346090    6040 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346096    6040 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:40.360689 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346100    6040 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346104    6040 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346106    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346109    6040 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346112    6040 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346114    6040 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346117    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346119    6040 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346122    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346124    6040 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346128    6040 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346130    6040 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346133    6040 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346136    6040 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346138    6040 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346141    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346144    6040 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346147    6040 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346149    6040 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:40.361168 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346152    6040 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346154    6040 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346157    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346160    6040 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346162    6040 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346166    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346170    6040 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346175    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346181    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.346205    6040 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346260    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346265    6040 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346270    6040 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346274    6040 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346277    6040 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346280    6040 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:40.361698 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346283    6040 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346286    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346289    6040 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346291    6040 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346294    6040 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346297    6040 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346300    6040 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346303    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346306    6040 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346309    6040 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346311    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346314    6040 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346317    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346319    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346324    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346329    6040 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346333    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346337    6040 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346340    6040 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346343    6040 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:40.362096 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346345    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346348    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346350    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346354    6040 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346359    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346361    6040 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346364    6040 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346366    6040 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346369    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346372    6040 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346375    6040 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346378    6040 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346380    6040 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346383    6040 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346386    6040 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346388    6040 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346391    6040 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346393    6040 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346396    6040 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:40.362566 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346399    6040 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346404    6040 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346408    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346412    6040 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346415    6040 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346418    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346421    6040 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346423    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346426    6040 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346428    6040 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346431    6040 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346434    6040 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346437    6040 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346439    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346441    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346444    6040 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346447    6040 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346449    6040 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346452    6040 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:40.363050 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346456    6040 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346461    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346463    6040 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346466    6040 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346469    6040 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346472    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346474    6040 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346478    6040 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346482    6040 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346487    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346491    6040 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346494    6040 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346496    6040 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346499    6040 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346502    6040 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346505    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346507    6040 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346510    6040 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346512    6040 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346514    6040 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:40.363533 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346517    6040 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346520    6040 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346522    6040 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346525    6040 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346527    6040 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346530    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346532    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346535    6040 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346537    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346541    6040 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346544    6040 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346546    6040 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346549    6040 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346551    6040 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346555    6040 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346559    6040 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346564    6040 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346572    6040 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346575    6040 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:40.364095 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346578    6040 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346580    6040 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:40.346583    6040 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.346636    6040 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.347605    6040 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.347669    6040 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.347874    6040 server.go:1014] "Starting client certificate rotation"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.348013    6040 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.348047    6040 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-28 23:07:28 +0000 UTC" deadline="2028-03-31 14:17:12.35456601 +0000 UTC"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.348083    6040 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12447h3m32.006485025s"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.348344    6040 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.350469    6040 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.351897    6040 log.go:25] "Validated CRI v1 runtime API"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.353641    6040 log.go:25] "Validated CRI v1 image API"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.354475    6040 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 29 23:13:40.364537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.356435    6040 fs.go:135] Filesystem UUIDs: map[153ded04-b3e4-47c1-ac22-6c3834ae04da:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b0856242-c573-4082-86de-174e5385a837:/dev/nvme0n1p4]
Oct 29 23:13:40.364960 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.356451    6040 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/003e85709816678966b319fe60adb5ce0a64a71563dcc1ee5febfd0126b2130b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/003e85709816678966b319fe60adb5ce0a64a71563dcc1ee5febfd0126b2130b/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0c222a2193223b38a930acaf812a9a17d91987fc97b53582e03319c9b9109770/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0c222a2193223b38a930acaf812a9a17d91987fc97b53582e03319c9b9109770/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/13454da792cd8c93595b1b5ba188b093d14a7a55d808fdb35592464e3466011c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/13454da792cd8c93595b1b5ba188b093d14a7a55d808fdb35592464e3466011c/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2998256652bea035ba14fa892c488c98f5d860a3edf9ddb465ebf694608034b8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2998256652bea035ba14fa892c488c98f5d860a3edf9ddb465ebf694608034b8/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2b01b5effb98e4713c27a97ca01ebf87cb07fc46b39f4caa02ec9fd0bf385741/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2b01b5effb98e4713c27a97ca01ebf87cb07fc46b39f4caa02ec9fd0bf385741/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2c18767b52ddca68d5aea85ec40d8c544d926d002bffd0b251c586cb6d2b5311/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2c18767b52ddca68d5aea85ec40d8c544d926d002bffd0b251c586cb6d2b5311/userdata/shm major:0 minor:359 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2e66982e28329514a8e12f28e3a3da95f22494b7a28b26fb3ec77944d0fdd0f1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2e66982e28329514a8e12f28e3a3da95f22494b7a28b26fb3ec77944d0fdd0f1/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/52da2c7c5f90348c938260e89583f55a50e3e9ac6d9223f47e1ca3f8886cefa7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/52da2c7c5f90348c938260e89583f55a50e3e9ac6d9223f47e1ca3f8886cefa7/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5bdb8aa5c502505aee9639e14dbb739a31238777723dbee7f9e274f97570f86c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5bdb8aa5c502505aee9639e14dbb739a31238777723dbee7f9e274f97570f86c/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6b450ddcc054ff72ae223cb160cfaf267a5093104ac64e78998c8ac0553345f3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6b450ddcc054ff72ae223cb160cfaf267a5093104ac64e78998c8ac0553345f3/userdata/shm major:0 minor:213 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6ee0426bf15653002e95687e82fc07c34e8202073a2f5cc46e68dda74aefd9e1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6ee0426bf15653002e95687e82fc07c34e8202073a2f5cc46e68dda74aefd9e1/userdata/shm major:0 minor:361 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7345dd53d558203baf30a745aa105d73e0a53547a2aaf7043638d52b28f30bae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7345dd53d558203baf30a745aa105d73e0a53547a2aaf7043638d52b28f30bae/userdata/shm major:0 minor:352 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/76edbc97b64d77cc4eb3bd88ad6e43e5ab4f389627af1d1d293c28c93aeb7d4b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/76edbc97b64d77cc4eb3bd88ad6e43e5ab4f389627af1d1d293c28c93aeb7d4b/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/933e890befee0f4c409fe216775374657a533bb4257324c695ea765223e96eb6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/933e890befee0f4c409fe216775374657a533bb4257324c695ea765223e96eb6/userdata/shm major:0 minor:317 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/952140ee592fcad7e773a0943888f1835b17a48d76037ccd3ef8392955128cd9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/952140ee592fcad7e773a0943888f1835b17a48d76037ccd3ef8392955128cd9/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b1499e2c5e1aa06e2198d073e65da4ab6655f5900a0e182f9d725660a5342897/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b1499e2c5e1aa06e2198d073e65da4ab6655f5900a0e182f9d725660a5342897/userdata/shm major:0 minor:285 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b343f5d0aa975e046424b0355a0a0f82af4d1e5d363fed0f128c60fac915fd40/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b343f5d0aa975e046424b0355a0a0f82af4d1e5d363fed0f128c60fac915fd40/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bbcdc18091b22a47939271a133581135955337afb04756ce9d6709a4521e29b8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bbcdc18091b22a47939271a133581135955337afb04756ce9d6709a4521e29b8/userdata/shm major:0 minor:247 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cee92baeb3c543a6802e899a973d7a47d97f488c26881ab311903320263e3cb0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cee92baeb3c543a6802e899a973d7a47d97f488c26881ab311903320263e3cb0/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d5b0afdb5cf53716a257e3d3008322004dde10191eb4e0005e53f73199063f3f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d5b0afdb5cf53716a257e3d3008322004dde10191eb4e0005e53f73199063f3f/userdata/shm major:0 minor:362 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fe3ca91ce0d3cfb9bee44ad3ce416052f3d41f3d294ad730c17496a89ee4a0b9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fe3ca91ce0d3cfb9bee44ad3ce416052f3d41f3d294ad730c17496a89ee4a0b9/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/22013da1-6ec7-4b6d-95f8-298e97fb1372/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/22013da1-6ec7-4b6d-95f8-298e97fb1372/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/22013da1-6ec7-4b6d-95f8-298e97fb1372/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/22013da1-6ec7-4b6d-95f8-298e97fb1372/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/22013da1-6ec7-4b6d-95f8-298e97fb1372/volumes/kubernetes.io~projected/kube-api-access-mxlfh:{mountpoint:/var/lib/kubelet/pods/22013da1-6ec7-4b6d-95f8-298e97fb1372/volumes/kubernetes.io~projected/kube-api-access-mxlfh major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~projected/kube-api-access-gdgp9:{mountpoint:/var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~projected/kube-api-access-gdgp9 major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:301 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:302 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2bc4c683-845b-40f3-ac2e-2c785b221257/volumes/kubernetes.io~projected/kube-api-access-vpjpd:{mountpoint:/var/lib/kubelet/pods/2bc4c683-845b-40f3-ac2e-2c785b221257/volumes/kubernetes.io~projected/kube-api-access-vpjpd major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f949f77-5aa6-4443-a09b-6fed361a0d43/volumes/kubernetes.io~projected/kube-api-access-rfnrq:{mountpoint:/var/lib/kubelet/pods/3f949f77-5aa6-4443-a09b-6fed361a0d43/volumes/kubernetes.io~projected/kube-api-access-rfnrq major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/54081aac-51a7-47aa-8dab-a04c8d68ab5f/volumes/kubernetes.io~projected/kube-api-access-b2wfh:{mountpoint:/var/lib/kubelet/pods/54081aac-51a7-47aa-8dab-a04c8d68ab5f/volumes/kubernetes.io~projected/kube-api-access-b2wfh major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/59d798e8-802a-4a6a-8f5a-063d69972c13/volumes/kubernetes.io~projected/kube-api-access-vlpjn:{mountpoint:/var/lib/kubelet/pods/59d798e8-802a-4a6a-8f5a-063d69972c13/volumes/kubernetes.io~projected/kube-api-access-vlpjn major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/59d798e8-802a-4a6a-8f5a-063d69972c13/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/59d798e8-802a-4a6a-8f5a-063d69972c13/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/59d798e8-802a-4a6a-8f5a-063d69972c13/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/59d798e8-802a-4a6a-8f5a-063d69972c13/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62ec6811-5291-4963-b8f1-52a7ffdc50bc/volumes/kubernetes.io~projected/kube-api-access-gcxgr:{mountpoint:/var/lib/kubelet/pods/62ec6811-5291-4963-b8f1-52a7ffdc50bc/volumes/kubernetes.io~projected/kube-api-access-gcxgr major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62ec6811-5291-4963-b8f1-52a7ffdc50bc/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/62ec6811-5291-4963-b8f1-52a7ffdc50bc/volumes/kubernetes.io~secret/metrics-certs major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5/volumes/kubernetes.io~projected/kube-api-access-qk7h7:{mountpoint:/var/lib/kubelet/pods/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5/volumes/kubernetes.io~projected/kube-api-access-qk7h7 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/802fe00a-6d8f-483e-9fb5-2ea266688ad1/volumes/kubernetes.io~projected/kube-api-access-gwqjz:{mountpoint:/var/lib/kubelet/pods/802fe00a-6d8f-483e-9fb5-2ea266688ad1/volumes/kubernetes.io~projected/kube-api-access-gwqjz major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/802fe00a-6d8f-483e-9fb5-2ea266688ad1/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/802fe00a-6d8f-483e-9fb5-2ea266688ad1/volumes/kubernetes.io~secret/metrics-tls major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87b4272a-c0a8-4ff8-8b80-cf9d76552032/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/87b4272a-c0a8-4ff8-8b80-cf9d76552032/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87b4272a-c0a8-4ff8-8b80-cf9d76552032/volumes/kubernetes.io~projected/kube-api-access-ffjqr:{mountpoint:/var/lib/kubelet/pods/87b4272a-c0a8-4ff8-8b80-cf9d76552032/volumes/kubernetes.io~projected/kube-api-access-ffjqr major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87b4272a-c0a8-4ff8-8b80-cf9d76552032/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/87b4272a-c0a8-4ff8-8b80-cf9d76552032/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/916781d5-af66-4d17-ba3f-748bab236eb1/volumes/kubernetes.io~projected/kube-api-access-9lb7c:{mountpoint:/var/lib/kubelet/pods/916781d5-af66-4d17-ba3f-748bab236eb1/volumes/kubernetes.io~projected/kube-api-access-9lb7c major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9eb8b8c3-aae3-4116-9d9e-244d34ba0271/volumes/kubernetes.io~projected/kube-api-access-8c5z9:{mountpoint:/var/lib/kubelet/pods/9eb8b8c3-aae3-4116-9d9e-244d34ba0271/volumes/kubernetes.io~projected/kube-api-access-8c5z9 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9eb8b8c3-aae3-4116-9d9e-244d34ba0271/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/9eb8b8c3-aae3-4116-9d9e-244d34ba0271/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~projected/kube-api-access-2qf9c:{mountpoint:/var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~projected/kube-api-access-2qf9c major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/web-config/alertmanager/9 major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/web-config/config-reloader/6 major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~empty-dir/config-out major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~projected/kube-api-access-pwpxg:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~projected/kube-api-access-pwpxg major:0 minor:244 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~projected/tls-assets major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/config-volume major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:242 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:239 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/web-config major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed669269-4708-416f-8b97-9ee2cc5bf891/volumes/kubernetes.io~projected/kube-api-access-xnw2k:{mountpoint:/var/lib/kubelet/pods/ed669269-4708-416f-8b97-9ee2cc5bf891/volumes/kubernetes.io~projected/kube-api-access-xnw2k major:0 minor:348 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eeeba32d-22c4-4173-83f0-5b4c391644b0/volumes/kubernetes.io~projected/kube-api-access-jpcwp:{mountpoint:/var/lib/kubelet/pods/eeeba32d-22c4-4173-83f0-5b4c391644b0/volumes/kubernetes.io~projected/kube-api-access-jpcwp major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f4ef6497-03ed-4ed8-beb9-1cde1f435655/volumes/kubernetes.io~projected/kube-api-access-tfhnw:{mountpoint:/var/lib/kubelet/pods/f4ef6497-03ed-4ed8-beb9-1cde1f435655/volumes/kubernetes.io~projected/kube-api-access-tfhnw major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f4ef6497-03ed-4ed8-beb9-1cde1f435655/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/f4ef6497-03ed-4ed8-beb9-1cde1f435655/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fd1c395e-bc72-4038-80cc-4ab131d96fdb/volumes/kubernetes.io~projected/kube-api-access-jbssw:{mountpoint:/var/lib/kubelet/pods/fd1c395e-bc72-4038-80cc-4ab131d96fdb/volumes/kubernetes.io~projected/kube-api-access-jbssw major:0 minor:358 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/fdb4ff45c4f18cf66336b68052e0e549730452366ae79df2bcd74c97103af85e/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/f74b510050ab2e8f43ebc9410adad2da143026364cb306cc935bc02ca015744c/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/dc28a731b3523a33dab057a7bc890587f10eb85f9c8346c51ca7abeec713e30f/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/20085e8770638af4f3499246aab8020a151c2bc1bd73627036dd8189d2a37063/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/4d96349854a141287cbc96c6bdfb0090221e8f748214166403bf065b5d9de423/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/7755288ee1e9b5b07874b40a237a095c630e342d4c304616943683a8d4d7becf/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/ae4a249e5f3a6ac9666ffd45d73ac35ef54acb67f0a064d2e361dd46c5abebd3/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/6d95ce1853ec4ea3e28d20e5e77a07827aa2e75b3e6ee91f95088317f9e3ebb0/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/27145914ff6a5c31bcb6e239829b0b71235c651cff5eb4014eda9bc254a3f88a/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/a127e0fc6fb24a01d545c06096b30a8da493b72f113e4f1e606a14cf630a1dff/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/cc1b5c25cb0ae316e5b1e7c01372fc5e5c35ef054024d57f6c7c2f42bcf75540/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-148:{mountpoint:/var/lib/containers/storage/overlay/91f91166063781eccf79244353a32a1e02b3fda174d0b72ac0b8c83d58ff4f49/merged major:0 minor:148 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/06cc34074771f100e2eca7949ec0aef8791ffb08f709179b6d79b382d9d2d0b0/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/4c98c9181da256e7c7bad5c5d761fcf4258c928c9449747f3c165cddb21b25ff/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/62ed6663b6ca1e22d53293cc5b844ea46a235d8fe5a88eb0cebf43ff88efc5e7/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/f5765dda687157c8028e414964b2c80c13979119a6ed1a1dc67c21a749b92641/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/2367750b1eaebf87e31a88d17660f5a01d29cc9f742c474acb0620b97cae83ae/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/39928893eca4a7b27912c163c535eedf5734172157ad450fefb6ac77ab0c2088/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/3b6fa09029ad91cd12c6b6129e4598e74fbd1a578258c00fb58baf9d27691931/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/10e4f15936aaa84b2f2d42046fe7c3084829b70b4984ad4eee03bda7804f17b9/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/097843adc5d0be69ed68390bdceca8fedbb76740eab78543afa8ad03d92c74b3/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-209:{mountpoint:/var/lib/containers/storage/overlay/078361294c6741999e4744e12cbfa01088b015619ee5d89090f25b2eeff68558/merged major:0 minor:209 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/e85d05577742afbe064b2d017b57fa543de2feedaedb8fafb5980d852c2ee3eb/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/731a561164de71771438f33d6d350b4688874d4948c4afbccdad2b7e03bdae83/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-221:{mountpoint:/var/lib/containers/storage/overlay/67393fd65645479245eb3026765c9b492f6e5f9bca1dc98133e7705ff06852df/merged major:0 minor:221 fsType:overlay blockSize:0} overlay_0-226:{mountpoint:/var/lib/containers/storage/overlay/9d2258b38041a2bbd1b2910e82977beeb4f1046aca5483bad4edffa553ce10a6/merged major:0 minor:226 fsType:overlay blockSize:0} overlay_0-233:{mountpoint:/var/lib/containers/storage/overlay/eb1a556323e93d976ac46a55ee4d85d90d4572592fc9c6d0006244a676570549/merged major:0 minor:233 fsType:overlay blockSize:0} overlay_0-245:{mountpoint:/var/lib/containers/storage/overlay/8c514539702b87e6945ede100d474d188d474058e2cbd89cf4d217f08f334c9b/merged major:0 minor:245 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/f8505609aa016403b86c568a0dcb0af0035bb7ce2e672440f68b7c5caa79c3c3/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/f656cf870ac1b2856d09ad74c64599e4101d90cc90e6ca6e10c1d47e88da0485/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/e17113e161df00da5089b89142f76ba196108b33fd63f9f55548d5f5dce2d52f/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/b66935a102fe8937448681623eec69e48f8a26605eee898ad6eb63329d742cba/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/3c2deeb596ef2eda362e88d2a27ee6ce0795affbff55387e9ceec702c3f5e5fd/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/6ed44eb0ce3b4dbf9daf46a140ee53a2af385bee3c65340d42878d6ca204b00d/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-287:{mountpoint:/var/lib/containers/storage/overlay/84436ebb3af0ea36935047977f4c148874bb541d179bfc1a89eca845b17d2b0c/merged major:0 minor:287 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/5a0a6dc01e99bfd99976bed53fd38593be151f77dcafe212934274572f3a8832/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-291:{mountpoint:/var/lib/containers/storage/overlay/212cc260558adcacd82db6f39db500eec6ad38a63d7ad6e2baf33ddfe620a0b8/merged major:0 minor:291 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/ea9aba40d2f36194505bb6089cc0fe54ea127a33f7da2488624400df65e25412/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/14d6fac9013a30defcb430a553e9fafed5723750494d687ec7057fec1ecb8eec/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/49a664dc0f2337738dbc2399a4f41bf0ca224f9fa144e6c6674c9c9f7a54f91b/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/c0b9bbe4c22dbcfcbdb30edb3ba8cbfb83c89f260ce0571df677ae76551018e8/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-324:{mountpoint:/var/lib/containers/storage/overlay/04ddf97246f7a789aacf4cf0349d03ce3bc215e278ef1e1024334cbda78ed5bb/merged major:0 minor:324 fsType:overlay blockSize:0} overlay_0-326:{mountpoint:/var/lib/containers/storage/overlay/f47e31806883919889b8b54244ffc5a33d01fd44d2a479a607b3690abd042c31/merged major:0 minor:326 fsType:overlay blockSize:0} overlay_0-331:{mountpoint:/var/lib/containers/storage/overlay/145e7cc6cc51663381be1bf565171587b98917d720af97f8b6b654798b48a27c/merged major:0 minor:331 fsType:overlay blockSize:0} overlay_0-337:{mountpoint:/var/lib/containers/storage/overlay/ce8bdcc93ed0979294ecfeccfa25634f87ca858d0b3e0a72c35ef051e6a8b3da/merged major:0 minor:337 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/2382d96ed3c19fb26b606ab885a3bcb5881a14a5f4b46709c95ee9fc244e507b/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/62e914be42029148b403c93c1913f98de9cb1c9a06c105c01c0c73c7c2bfcc7b/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-365:{mountpoint:/var/lib/containers/storage/overlay/1e613135a7abb6a4a9b4a8739c34bd520db9c9204a773495e596572d7ee88968/merged major:0 minor:365 fsType:overlay blockSize:0} overlay_0-367:{mountpoint:/var/lib/containers/storage/overlay/e1f473d7416248c37b6ed5e668e400618b7bae8e09622b97b1f99d2b15bacb4a/merged major:0 minor:367 fsType:overlay blockSize:0} overlay_0-369:{mountpoint:/var/lib/containers/storage/overlay/7e8eb4b836ff39182401acd962a18394d60b1d40da1e733419fcb803de532392/merged major:0 minor:369 fsType:overlay blockSize:0} overlay_0-371:{mountpoint:/var/lib/containers/storage/overlay/2ae871fe0d256e8e9005be298e37e6d56f754582565f6f604b486a99e5a4f2c7/merged major:0 minor:371 fsType:overlay blockSize:0} overlay_0-373:{mountpoint:/var/lib/containers/storage/overlay/6c00bc7c794a4e1b2721688e152033f703a93c5599a8b52f0d4f5ef65e7918d1/merged major:0 minor:373 fsType:overlay blockSize:0} overlay_0-379:{mountpoint:/var/lib/containers/storage/overlay/f355ebfbc0bf83f79a8be8104cd6dd90a448ba468d5ee4a23720be86f36c3aca/merged major:0 minor:379 fsType:overlay blockSize:0} overlay_0-384:{mountpoint:/var/lib/containers/storage/overlay/3dd168371a764996921cf3357d2da7a8e9699eef26ae7ac758e12f0fd829f784/merged major:0 minor:384 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/c215b19c2ea89171f522ae0144c5a9affbf0b82bd29a385009f151d3812d3632/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/2ab02fb5ac4e2d04aac2ece95e858eefd7bd30838ceb4fec5a6db57043e4d7ac/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/416010dc8aeeb59284661ec8118d9af6aea62c85eace032f4eb485215d0f70da/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/74b0271cb88747f24f53b898e5509b87a900faf497c0e5ee452566dfd6300dd2/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/377ad48f75f85405c213854a040b033d7d36c54e8922cb162b303f8b42305052/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/cf333e8f5cc55fb8880298847422d2cf7490d02c57d4aa4d05fd498f1563c6cf/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/5a9c3ec1a3a9eeaa79bf7d2fa81c01be6deec6c6e1e59e775a353ef0002c92b8/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/ba9f2cd689faf4c197d49383c9ea6ffcc77e7d7336d2b6e3a5f7b92a986eff11/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/45d511994d84663d2b51d26ab8844b8c3edaf59cfca2ff35338556a43acf995a/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/e686c1b059bded73c9b6d011ef32ec8647a11c81290d9b8797287dfc01ab404a/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 29 23:13:40.365180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.364260    6040 manager.go:217] Machine: {Timestamp:2026-10-29 23:13:40.363802545 +0000 UTC m=+0.058911415 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099614 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec239fb954fbcd98fb75b56338fa5772 SystemUUID:ec239fb9-54fb-cd98-fb75-b56338fa5772 BootID:bfc8ceae-1f27-4652-a4d4-ffba5207a5c0 Filesystems:[{Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-324 DeviceMajor:0 DeviceMinor:324 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/87b4272a-c0a8-4ff8-8b80-cf9d76552032/volumes/kubernetes.io~projected/kube-api-access-ffjqr DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/952140ee592fcad7e773a0943888f1835b17a48d76037ccd3ef8392955128cd9/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/87b4272a-c0a8-4ff8-8b80-cf9d76552032/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-221 DeviceMajor:0 DeviceMinor:221 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-287 DeviceMajor:0 DeviceMinor:287 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/22013da1-6ec7-4b6d-95f8-298e97fb1372/volumes/kubernetes.io~projected/kube-api-access-mxlfh DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-209 DeviceMajor:0 DeviceMinor:209 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/933e890befee0f4c409fe216775374657a533bb4257324c695ea765223e96eb6/userdata/shm DeviceMajor:0 DeviceMinor:317 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7345dd53d558203baf30a745aa105d73e0a53547a2aaf7043638d52b28f30bae/userdata/shm DeviceMajor:0 DeviceMinor:352 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/59d798e8-802a-4a6a-8f5a-063d69972c13/volumes/kubernetes.io~projected/kube-api-access-vlpjn DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:235 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/62ec6811-5291-4963-b8f1-52a7ffdc50bc/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:356 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6ee0426bf15653002e95687e82fc07c34e8202073a2f5cc46e68dda74aefd9e1/userdata/shm DeviceMajor:0 DeviceMinor:361 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2bc4c683-845b-40f3-ac2e-2c785b221257/volumes/kubernetes.io~projected/kube-api-access-vpjpd DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f4ef6497-03ed-4ed8-beb9-1cde1f435655/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:242 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:301 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eeeba32d-22c4-4173-83f0-5b4c391644b0/volumes/kubernetes.io~projected/kube-api-access-jpcwp DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:237 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:303 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-337 DeviceMajor:0 DeviceMinor:337 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-367 DeviceMajor:0 DeviceMinor:367 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fe3ca91ce0d3cfb9bee44ad3ce416052f3d41f3d294ad730c17496a89ee4a0b9/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d5b0afdb5cf53716a257e3d3008322004dde10191eb4e0005e53f73199063f3f/userdata/shm DeviceMajor:0 DeviceMinor:362 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6b450ddcc054ff72ae223cb160cfaf267a5093104ac64e78998c8ac0553345f3/userdata/shm DeviceMajor:0 DeviceMinor:213 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:238 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-379 DeviceMajor:0 DeviceMinor:379 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3f949f77-5aa6-4443-a09b-6fed361a0d43/volumes/kubernetes.io~projected/kube-api-access-rfnrq DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:236 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:238 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:240 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:240 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fd1c395e-bc72-4038-80cc-4ab131d96fdb/volumes/kubernetes.io~projected/kube-api-access-jbssw DeviceMajor:0 DeviceMinor:358 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2c18767b52ddca68d5aea85ec40d8c544d926d002bffd0b251c586cb6d2b5311/userdata/shm DeviceMajor:0 DeviceMinor:359 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5bdb8aa5c502505aee9639e14dbb739a31238777723dbee7f9e274f97570f86c/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9eb8b8c3-aae3-4116-9d9e-244d34ba0271/volumes/kubernetes.io~projected/kube-api-access-8c5z9 DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/802fe00a-6d8f-483e-9fb5-2ea266688ad1/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:205 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/62ec6811-5291-4963-b8f1-52a7ffdc50bc/volumes/kubernetes.io~projected/kube-api-access-gcxgr DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:282 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/916781d5-af66-4d17-ba3f-748bab236eb1/volumes/kubernetes.io~projected/kube-api-access-9lb7c DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cee92baeb3c543a6802e899a973d7a47d97f488c26881ab311903320263e3cb0/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b343f5d0aa975e046424b0355a0a0f82af4d1e5d363fed0f128c60fac915fd40/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/52da2c7c5f90348c938260e89583f55a50e3e9ac6d9223f47e1ca3f8886cefa7/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/22013da1-6ec7-4b6d-95f8-298e97fb1372/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:277 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/003e85709816678966b319fe60adb5ce0a64a71563dcc1ee5febfd0126b2130b/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/87b4272a-c0a8-4ff8-8b80-cf9d76552032/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/13454da792cd8c93595b1b5ba188b093d14a7a55d808fdb35592464e3466011c/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:240 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~projected/kube-api-access-gdgp9 DeviceMajor:0 DeviceMinor:297 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ed669269-4708-416f-8b97-9ee2cc5bf891/volumes/kubernetes.io~projected/kube-api-access-xnw2k DeviceMajor:0 DeviceMinor:348 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f4ef6497-03ed-4ed8-beb9-1cde1f435655/volumes/kubernetes.io~projected/kube-api-access-tfhnw DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/59d798e8-802a-4a6a-8f5a-063d69972c13/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:224 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9eb8b8c3-aae3-4116-9d9e-244d34ba0271/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-245 DeviceMajor:0 DeviceMinor:245 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:239 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-148 DeviceMajor:0 DeviceMinor:148 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-365 DeviceMajor:0 DeviceMinor:365 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:241 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~projected/kube-api-access-pwpxg DeviceMajor:0 DeviceMinor:244 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-373 DeviceMajor:0 DeviceMinor:373 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bbcdc18091b22a47939271a133581135955337afb04756ce9d6709a4521e29b8/userdata/shm DeviceMajor:0 DeviceMinor:247 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b1499e2c5e1aa06e2198d073e65da4ab6655f5900a0e182f9d725660a5342897/userdata/shm DeviceMajor:0 DeviceMinor:285 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:357 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-371 DeviceMajor:0 DeviceMinor:371 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/59d798e8-802a-4a6a-8f5a-063d69972c13/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:223 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-226 DeviceMajor:0 DeviceMinor:226 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2b01b5effb98e4713c27a97ca01ebf87cb07fc46b39f4caa02ec9fd0bf385741/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2e66982e28329514a8e12f28e3a3da95f22494b7a28b26fb3ec77944d0fdd0f1/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:243 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/22013da1-6ec7-4b6d-95f8-298e97fb1372/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/802fe00a-6d8f-483e-9fb5-2ea266688ad1/volumes/kubernetes.io~projected/kube-api-access-gwqjz DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:302 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-384 DeviceMajor:0 DeviceMinor:384 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-233 DeviceMajor:0 DeviceMinor:233 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-369 DeviceMajor:0 DeviceMinor:369 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/54081aac-51a7-47aa-8dab-a04c8d68ab5f/volumes/kubernetes.io~projected/kube-api-access-b2wfh DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2998256652bea035ba14fa892c488c98f5d860a3edf9ddb465ebf694608034b8/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:240 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~projected/kube-api-access-2qf9c DeviceMajor:0 DeviceMinor:284 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/259771b7-e14a-496a-bba1-a8f7364e6c1f/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:304 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/76edbc97b64d77cc4eb3bd88ad6e43e5ab4f389627af1d1d293c28c93aeb7d4b/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cbf0e826-b477-47a2-9c6d-08ac105aedd0/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:283 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-291 DeviceMajor:0 DeviceMinor:291 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-326 DeviceMajor:0 DeviceMinor:326 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-331 DeviceMajor:0 DeviceMinor:331 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5/volumes/kubernetes.io~projected/kube-api-access-qk7h7 DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0c222a2193223b38a930acaf812a9a17d91987fc97b53582e03319c9b9109770/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0c222a2193223b3 MacAddress:1a:87:30:34:33:4b Speed:10000 Mtu:8901} {Name:2998256652bea03 MacAddress:62:d9:20:76:c6:17 Speed:10000 Mtu:8901} {Name:2c18767b52ddca6 MacAddress:c6:e3:19:8a:57:a7 Speed:10000 Mtu:8901} {Name:6b450ddcc054ff7 MacAddress:aa:9d:02:02:f1:05 Speed:10000 Mtu:8901} {Name:6ee0426bf156530 MacAddress:62:73:f0:1f:56:82 Speed:10000 Mtu:8901} {Name:7345dd53d558203 MacAddress:46:cc:9e:14:08:3c Speed:10000 Mtu:8901} {Name:933e890befee0f4 MacAddress:3e:a6:82:32:15:b5 Speed:10000 Mtu:8901} {Name:b1499e2c5e1aa06 MacAddress:5a:5a:5c:64:7d:e1 Speed:10000 Mtu:8901} {Name:bbcdc18091b22a4 MacAddress:7a:fb:16:8c:a8:3b Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:f4:56:1c:3b Speed:0 Mtu:9001} {Name:br-int MacAddress:56:c4:d8:02:be:06 Speed:0 Mtu:8901} {Name:d5b0afdb5cf5371 MacAddress:e2:d2:0e:85:07:b9 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:f4:56:1c:3b Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:76:3c:89:1f:c9:a5 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:96:ba:59:b7:bf:5f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 29 23:13:40.365180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.364608    6040 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 29 23:13:40.365180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.364650    6040 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 29 23:13:40.365180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.364935    6040 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 29 23:13:40.365180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.364955    6040 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-189.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 29 23:13:40.365180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365076    6040 topology_manager.go:143] "Creating topology manager with none policy"
Oct 29 23:13:40.365180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365083    6040 container_manager_linux.go:311] "Creating device plugin manager"
Oct 29 23:13:40.365180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365093    6040 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 29 23:13:40.365180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365113    6040 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 29 23:13:40.365550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365188    6040 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 29 23:13:40.365550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365337    6040 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 29 23:13:40.365550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365451    6040 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 29 23:13:40.365550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365484    6040 kubelet.go:493] "Attempting to sync node with API server"
Oct 29 23:13:40.365550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365497    6040 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 29 23:13:40.365550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365507    6040 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 29 23:13:40.365550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365516    6040 kubelet.go:405] "Adding apiserver pod source"
Oct 29 23:13:40.365550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.365524    6040 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 29 23:13:40.366393 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.366366    6040 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 29 23:13:40.366393 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.366382    6040 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 29 23:13:40.366970 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.366935    6040 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 29 23:13:40.375706 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.375688    6040 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 29 23:13:40.375706 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.375703    6040 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 29 23:13:40.375839 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.375825    6040 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 29 23:13:40.376015 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376003    6040 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 29 23:13:40.376045 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376033    6040 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 29 23:13:40.376130 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376121    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 29 23:13:40.376159 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376138    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 29 23:13:40.376159 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376148    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 29 23:13:40.376159 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376156    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 29 23:13:40.376241 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376162    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 29 23:13:40.376241 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376168    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 29 23:13:40.376241 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376174    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 29 23:13:40.376241 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376179    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 29 23:13:40.376241 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376187    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 29 23:13:40.376241 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376193    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 29 23:13:40.376241 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376207    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 29 23:13:40.376241 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376217    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 29 23:13:40.376241 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376240    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 29 23:13:40.376495 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.376248    6040 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 29 23:13:40.379536 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.379523    6040 server.go:1263] "Started kubelet"
Oct 29 23:13:40.379649 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.379593    6040 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 29 23:13:40.379700 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.379621    6040 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 29 23:13:40.379700 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.379693    6040 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 29 23:13:40.380063 ip-10-0-138-189 systemd[1]: Started Kubernetes Kubelet.
Oct 29 23:13:40.380253 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.380146    6040 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 29 23:13:40.380958 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.380929    6040 server.go:317] "Adding debug handlers to kubelet server"
Oct 29 23:13:40.383947 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.383929    6040 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 29 23:13:40.384126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.384109    6040 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 29 23:13:40.384126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.384119    6040 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 29 23:13:40.384126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.384127    6040 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.384217    6040 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:40.384333    6040 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-189.ec2.internal\" not found"
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.384389    6040 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 23:07:30 +0000 UTC" deadline="2028-06-11 05:14:41.983005023 +0000 UTC"
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.384423    6040 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14166h1m1.598585365s"
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.384827    6040 factory.go:55] Registering systemd factory
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.384845    6040 factory.go:223] Registration of the systemd container factory successfully
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.385286    6040 factory.go:153] Registering CRI-O factory
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.385307    6040 factory.go:223] Registration of the crio container factory successfully
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.385372    6040 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.385395    6040 factory.go:103] Registering Raw factory
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.385415    6040 manager.go:1199] Started watching for new ooms in manager
Oct 29 23:13:40.386043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.385941    6040 manager.go:319] Starting recovery of all containers
Oct 29 23:13:40.389055 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389022    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62ec6811-5291-4963-b8f1-52a7ffdc50bc" volumeName="kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs" seLinuxMountContext=""
Oct 29 23:13:40.389142 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389064    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2bc4c683-845b-40f3-ac2e-2c785b221257" volumeName="kubernetes.io/projected/2bc4c683-845b-40f3-ac2e-2c785b221257-kube-api-access-vpjpd" seLinuxMountContext=""
Oct 29 23:13:40.389142 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389079    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="59d798e8-802a-4a6a-8f5a-063d69972c13" volumeName="kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 29 23:13:40.389142 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389097    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87b4272a-c0a8-4ff8-8b80-cf9d76552032" volumeName="kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-env-overrides" seLinuxMountContext=""
Oct 29 23:13:40.389142 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389109    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="916781d5-af66-4d17-ba3f-748bab236eb1" volumeName="kubernetes.io/projected/916781d5-af66-4d17-ba3f-748bab236eb1-kube-api-access-9lb7c" seLinuxMountContext=""
Oct 29 23:13:40.389142 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389123    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9eb8b8c3-aae3-4116-9d9e-244d34ba0271" volumeName="kubernetes.io/secret/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-cert" seLinuxMountContext=""
Oct 29 23:13:40.389142 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389140    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="59d798e8-802a-4a6a-8f5a-063d69972c13" volumeName="kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389155    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="916781d5-af66-4d17-ba3f-748bab236eb1" volumeName="kubernetes.io/configmap/916781d5-af66-4d17-ba3f-748bab236eb1-serviceca" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389188    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8ac2ba2-0d4b-4498-a7c6-658412d60ccf" volumeName="kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389202    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389220    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4ef6497-03ed-4ed8-beb9-1cde1f435655" volumeName="kubernetes.io/empty-dir/f4ef6497-03ed-4ed8-beb9-1cde1f435655-data-volume" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389234    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd1c395e-bc72-4038-80cc-4ab131d96fdb" volumeName="kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389248    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9eb8b8c3-aae3-4116-9d9e-244d34ba0271" volumeName="kubernetes.io/projected/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-kube-api-access-8c5z9" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389273    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22013da1-6ec7-4b6d-95f8-298e97fb1372" volumeName="kubernetes.io/projected/22013da1-6ec7-4b6d-95f8-298e97fb1372-kube-api-access-mxlfh" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389293    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="259771b7-e14a-496a-bba1-a8f7364e6c1f" volumeName="kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-federate-client-tls" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389311    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="259771b7-e14a-496a-bba1-a8f7364e6c1f" volumeName="kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-client-tls" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389326    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbf0e826-b477-47a2-9c6d-08ac105aedd0" volumeName="kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389344    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4ef6497-03ed-4ed8-beb9-1cde1f435655" volumeName="kubernetes.io/projected/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-api-access-tfhnw" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389357    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="259771b7-e14a-496a-bba1-a8f7364e6c1f" volumeName="kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389371    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="59d798e8-802a-4a6a-8f5a-063d69972c13" volumeName="kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-metrics-client-ca" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389388    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="59d798e8-802a-4a6a-8f5a-063d69972c13" volumeName="kubernetes.io/empty-dir/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-textfile" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389402    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="59d798e8-802a-4a6a-8f5a-063d69972c13" volumeName="kubernetes.io/projected/59d798e8-802a-4a6a-8f5a-063d69972c13-kube-api-access-vlpjn" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389419    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed669269-4708-416f-8b97-9ee2cc5bf891" volumeName="kubernetes.io/projected/ed669269-4708-416f-8b97-9ee2cc5bf891-kube-api-access-xnw2k" seLinuxMountContext=""
Oct 29 23:13:40.389434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389432    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4ef6497-03ed-4ed8-beb9-1cde1f435655" volumeName="kubernetes.io/configmap/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389446    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-metrics-client-ca" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389464    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="54081aac-51a7-47aa-8dab-a04c8d68ab5f" volumeName="kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389478    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbf0e826-b477-47a2-9c6d-08ac105aedd0" volumeName="kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389496    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbf0e826-b477-47a2-9c6d-08ac105aedd0" volumeName="kubernetes.io/empty-dir/cbf0e826-b477-47a2-9c6d-08ac105aedd0-audit-log" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389522    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22013da1-6ec7-4b6d-95f8-298e97fb1372" volumeName="kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-tmp" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389536    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="259771b7-e14a-496a-bba1-a8f7364e6c1f" volumeName="kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389551    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5" volumeName="kubernetes.io/configmap/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-iptables-alerter-script" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389575    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed669269-4708-416f-8b97-9ee2cc5bf891" volumeName="kubernetes.io/empty-dir/ed669269-4708-416f-8b97-9ee2cc5bf891-tmp" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389589    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="259771b7-e14a-496a-bba1-a8f7364e6c1f" volumeName="kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389608    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9271cfd4-96c6-46a0-84c9-dcdcdccaae4e" volumeName="kubernetes.io/configmap/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-konnectivity-ca" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389619    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9271cfd4-96c6-46a0-84c9-dcdcdccaae4e" volumeName="kubernetes.io/secret/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-agent-certs" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389631    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-main-db" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389651    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="259771b7-e14a-496a-bba1-a8f7364e6c1f" volumeName="kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-metrics-client-ca" seLinuxMountContext=""
Oct 29 23:13:40.390211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.389665    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="54081aac-51a7-47aa-8dab-a04c8d68ab5f" volumeName="kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 29 23:13:40.390581 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.390306    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="802fe00a-6d8f-483e-9fb5-2ea266688ad1" volumeName="kubernetes.io/projected/802fe00a-6d8f-483e-9fb5-2ea266688ad1-kube-api-access-gwqjz" seLinuxMountContext=""
Oct 29 23:13:40.390581 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.390341    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 29 23:13:40.390581 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.390356    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4ef6497-03ed-4ed8-beb9-1cde1f435655" volumeName="kubernetes.io/secret/f4ef6497-03ed-4ed8-beb9-1cde1f435655-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 29 23:13:40.390581 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.390377    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62ec6811-5291-4963-b8f1-52a7ffdc50bc" volumeName="kubernetes.io/projected/62ec6811-5291-4963-b8f1-52a7ffdc50bc-kube-api-access-gcxgr" seLinuxMountContext=""
Oct 29 23:13:40.391243 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.390390    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="802fe00a-6d8f-483e-9fb5-2ea266688ad1" volumeName="kubernetes.io/configmap/802fe00a-6d8f-483e-9fb5-2ea266688ad1-config-volume" seLinuxMountContext=""
Oct 29 23:13:40.391324 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391258    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="802fe00a-6d8f-483e-9fb5-2ea266688ad1" volumeName="kubernetes.io/empty-dir/802fe00a-6d8f-483e-9fb5-2ea266688ad1-tmp-dir" seLinuxMountContext=""
Oct 29 23:13:40.391324 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391280    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="802fe00a-6d8f-483e-9fb5-2ea266688ad1" volumeName="kubernetes.io/secret/802fe00a-6d8f-483e-9fb5-2ea266688ad1-metrics-tls" seLinuxMountContext=""
Oct 29 23:13:40.391324 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391295    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 29 23:13:40.391324 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391308    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 29 23:13:40.391324 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391321    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eeeba32d-22c4-4173-83f0-5b4c391644b0" volumeName="kubernetes.io/projected/eeeba32d-22c4-4173-83f0-5b4c391644b0-kube-api-access-jpcwp" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391335    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391350    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-cluster-tls-config" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391361    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eeeba32d-22c4-4173-83f0-5b4c391644b0" volumeName="kubernetes.io/empty-dir/eeeba32d-22c4-4173-83f0-5b4c391644b0-tmp-dir" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391374    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2bc4c683-845b-40f3-ac2e-2c785b221257" volumeName="kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-cni-binary-copy" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391387    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2bc4c683-845b-40f3-ac2e-2c785b221257" volumeName="kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-daemon-config" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391399    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbf0e826-b477-47a2-9c6d-08ac105aedd0" volumeName="kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-tls" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391412    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-tls-assets" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391426    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="259771b7-e14a-496a-bba1-a8f7364e6c1f" volumeName="kubernetes.io/projected/259771b7-e14a-496a-bba1-a8f7364e6c1f-kube-api-access-gdgp9" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391449    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f949f77-5aa6-4443-a09b-6fed361a0d43" volumeName="kubernetes.io/projected/3f949f77-5aa6-4443-a09b-6fed361a0d43-kube-api-access-rfnrq" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391462    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="54081aac-51a7-47aa-8dab-a04c8d68ab5f" volumeName="kubernetes.io/projected/54081aac-51a7-47aa-8dab-a04c8d68ab5f-kube-api-access-b2wfh" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391476    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="59d798e8-802a-4a6a-8f5a-063d69972c13" volumeName="kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-tls" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391488    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5" volumeName="kubernetes.io/projected/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-kube-api-access-qk7h7" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391500    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbf0e826-b477-47a2-9c6d-08ac105aedd0" volumeName="kubernetes.io/projected/cbf0e826-b477-47a2-9c6d-08ac105aedd0-kube-api-access-2qf9c" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391514    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="54081aac-51a7-47aa-8dab-a04c8d68ab5f" volumeName="kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-binary-copy" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391526    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87b4272a-c0a8-4ff8-8b80-cf9d76552032" volumeName="kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-config" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391539    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87b4272a-c0a8-4ff8-8b80-cf9d76552032" volumeName="kubernetes.io/projected/87b4272a-c0a8-4ff8-8b80-cf9d76552032-kube-api-access-ffjqr" seLinuxMountContext=""
Oct 29 23:13:40.391588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391573    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87b4272a-c0a8-4ff8-8b80-cf9d76552032" volumeName="kubernetes.io/secret/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 29 23:13:40.392399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391602    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-web-config" seLinuxMountContext=""
Oct 29 23:13:40.392399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391618    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbf0e826-b477-47a2-9c6d-08ac105aedd0" volumeName="kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 29 23:13:40.392399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391630    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbf0e826-b477-47a2-9c6d-08ac105aedd0" volumeName="kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-client-ca-bundle" seLinuxMountContext=""
Oct 29 23:13:40.392399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391645    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22013da1-6ec7-4b6d-95f8-298e97fb1372" volumeName="kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-tuned" seLinuxMountContext=""
Oct 29 23:13:40.392399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391657    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="259771b7-e14a-496a-bba1-a8f7364e6c1f" volumeName="kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 29 23:13:40.392399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391669    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87b4272a-c0a8-4ff8-8b80-cf9d76552032" volumeName="kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-script-lib" seLinuxMountContext=""
Oct 29 23:13:40.392399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391682    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-out" seLinuxMountContext=""
Oct 29 23:13:40.392399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391695    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-kube-api-access-pwpxg" seLinuxMountContext=""
Oct 29 23:13:40.392399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391722    6040 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf091e44-0c38-428f-a14f-f3a44f9b178b" volumeName="kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-volume" seLinuxMountContext=""
Oct 29 23:13:40.392399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391760    6040 reconstruct.go:97] "Volume reconstruction finished"
Oct 29 23:13:40.392399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.391768    6040 reconciler.go:29] "Reconciler: start to sync state"
Oct 29 23:13:40.415389 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.415351    6040 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 29 23:13:40.417357 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.417334    6040 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 29 23:13:40.417357 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.417362    6040 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 29 23:13:40.417556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.417382    6040 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 29 23:13:40.417556 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:40.417422    6040 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 29 23:13:40.436295 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.436268    6040 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:13:40.436833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.436812    6040 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 29 23:13:40.442081 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.442052    6040 generic.go:356] "Generic (PLEG): container finished" podID="aa79b04c65fcc2c4af3f0c1b29fc1ed6" containerID="2f656a59facaba97fbabae29c0751b97aff91da38d1bafb0db437d74f5a1848e" exitCode=0
Oct 29 23:13:40.444059 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.444031    6040 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:13:40.448055 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.448031    6040 generic.go:356] "Generic (PLEG): container finished" podID="59d798e8-802a-4a6a-8f5a-063d69972c13" containerID="52ec78d673ec09bc028c431ef2c529e76f1901bca8d98b290fb1900cc0021a00" exitCode=0
Oct 29 23:13:40.456226 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.456206    6040 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:13:40.456594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.456561    6040 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:13:40.461935 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.461913    6040 generic.go:356] "Generic (PLEG): container finished" podID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerID="0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66" exitCode=0
Oct 29 23:13:40.469654 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.469626    6040 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="ea993ffdfadf7e2a1281f3d4c035b8632c3a02d3eb8829d9e72bc85c2f9777e7" exitCode=0
Oct 29 23:13:40.469761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.469657    6040 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="bd3acb832876de97996dffe57f5fa336b66f2b0c926bd9470adab47842cd3c1f" exitCode=0
Oct 29 23:13:40.469761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.469671    6040 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="97e76f94125d393d0ed0187f84501998d3badc0f55990fe0ad69ebe5d8a37daf" exitCode=0
Oct 29 23:13:40.469761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.469681    6040 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="f7114c7a00efda02498591754056d2e74a509333ef6865718fc67a9f5d6933f8" exitCode=0
Oct 29 23:13:40.469761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.469690    6040 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="8ab70ab24c46b716274331f6a97f6a6a91e6b62362fd441370f327291212db0d" exitCode=0
Oct 29 23:13:40.469761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.469698    6040 generic.go:356] "Generic (PLEG): container finished" podID="54081aac-51a7-47aa-8dab-a04c8d68ab5f" containerID="d056999b16e340a2858c82fa1a599b9f88211e4f1886edfadb38e2e27c74382a" exitCode=0
Oct 29 23:13:40.483889 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.483841    6040 manager.go:324] Recovery completed
Oct 29 23:13:40.507406 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.507384    6040 cpu_manager.go:226] "Starting" policy="none"
Oct 29 23:13:40.507406 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.507400    6040 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 29 23:13:40.507535 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.507419    6040 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 29 23:13:40.507577 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.507536    6040 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 29 23:13:40.507577 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.507551    6040 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 29 23:13:40.507577 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.507566    6040 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 29 23:13:40.507577 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.507572    6040 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 29 23:13:40.507577 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.507577    6040 policy_none.go:50] "Start"
Oct 29 23:13:40.507757 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.507585    6040 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 29 23:13:40.507757 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.507594    6040 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 29 23:13:40.509142 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.509128    6040 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 29 23:13:40.509183 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.509142    6040 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 29 23:13:40.509183 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.509149    6040 policy_none.go:44] "Start"
Oct 29 23:13:40.514012 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.513517    6040 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 29 23:13:40.517519 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:40.517499    6040 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 29 23:13:40.519164 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.519150    6040 manager.go:341] "Starting Device Plugin manager"
Oct 29 23:13:40.519234 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:40.519179    6040 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 29 23:13:40.519234 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.519190    6040 server.go:87] "Starting device plugin registration server"
Oct 29 23:13:40.519491 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.519478    6040 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 29 23:13:40.519562 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.519495    6040 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 29 23:13:40.519615 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.519598    6040 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 29 23:13:40.519751 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.519695    6040 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 29 23:13:40.519809 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.519761    6040 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 29 23:13:40.519809 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.519769    6040 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 29 23:13:40.519944 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.519862    6040 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-29T23:13:40.519730785Z","UUID":"5b2cc035-a079-4446-9fd2-ad06818927c6","Handler":null,"Name":"","Endpoint":""}
Oct 29 23:13:40.521136 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.521120    6040 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 29 23:13:40.521215 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.521142    6040 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 29 23:13:40.521390 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:40.521363    6040 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 29 23:13:40.620511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.620480    6040 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:13:40.622418 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.622402    6040 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:13:40.622482 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.622432    6040 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:13:40.622482 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.622443    6040 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-189.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:13:40.622482 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.622481    6040 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.664226 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.664201    6040 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.664319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.664279    6040 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.718355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.718308    6040 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"]
Oct 29 23:13:40.719027 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.718991    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal" event={"ID":"2f6cf61a946e2452e67c50b8a0479c0d","Type":"ContainerStarted","Data":"b860ef0ce804bd310317385f99bf784bbe458a7663a3b71b327c4b708c5c6f7c"}
Oct 29 23:13:40.719027 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.719023    6040 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.719150 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.719032    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal" event={"ID":"2f6cf61a946e2452e67c50b8a0479c0d","Type":"ContainerStarted","Data":"003e85709816678966b319fe60adb5ce0a64a71563dcc1ee5febfd0126b2130b"}
Oct 29 23:13:40.719150 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.719065    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal" event={"ID":"aa79b04c65fcc2c4af3f0c1b29fc1ed6","Type":"ContainerStarted","Data":"3a36e5d7e20a7429043cf2e21dbb65cea88ab146a3715760f3171018fda1d764"}
Oct 29 23:13:40.719150 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.719080    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal" event={"ID":"aa79b04c65fcc2c4af3f0c1b29fc1ed6","Type":"ContainerDied","Data":"2f656a59facaba97fbabae29c0751b97aff91da38d1bafb0db437d74f5a1848e"}
Oct 29 23:13:40.719150 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.719094    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal" event={"ID":"aa79b04c65fcc2c4af3f0c1b29fc1ed6","Type":"ContainerStarted","Data":"5bdb8aa5c502505aee9639e14dbb739a31238777723dbee7f9e274f97570f86c"}
Oct 29 23:13:40.719292 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.719278    6040 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.747365 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.747313    6040 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:13:40.747365 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:40.747358    6040 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-189.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.748719 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.748708    6040 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:13:40.748778 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:40.748756    6040 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.839052 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.838986    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.839132 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.839057    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2f6cf61a946e2452e67c50b8a0479c0d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-189.ec2.internal\" (UID: \"2f6cf61a946e2452e67c50b8a0479c0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.839132 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.839081    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.939638 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.939608    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.939759 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.939645    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.939759 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.939707    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.939759 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.939712    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa79b04c65fcc2c4af3f0c1b29fc1ed6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" (UID: \"aa79b04c65fcc2c4af3f0c1b29fc1ed6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.939860 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.939777    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2f6cf61a946e2452e67c50b8a0479c0d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-189.ec2.internal\" (UID: \"2f6cf61a946e2452e67c50b8a0479c0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:40.939860 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:40.939814    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2f6cf61a946e2452e67c50b8a0479c0d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-189.ec2.internal\" (UID: \"2f6cf61a946e2452e67c50b8a0479c0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:41.366251 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.366217    6040 apiserver.go:52] "Watching apiserver"
Oct 29 23:13:41.416819 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.416788    6040 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 29 23:13:41.417433 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.417399    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-wht7s","openshift-multus/network-metrics-daemon-bv5tv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal","openshift-multus/multus-additional-cni-plugins-wmfk5","openshift-console/console-7c654d988-5jnqt","openshift-ingress-canary/ingress-canary-qlpqj","kube-system/konnectivity-agent-5z4zk","openshift-network-operator/iptables-alerter-vk7hb","openshift-dns/node-resolver-n4lkv","openshift-multus/multus-95nsq","openshift-console/downloads-6dd789c8b7-9jhw5","openshift-monitoring/metrics-server-78f9fd9c95-fj6xw","openshift-network-diagnostics/network-check-target-xhlf6","openshift-monitoring/telemeter-client-79699f4d76-w6hb5","openshift-insights/insights-runtime-extractor-fk7ml","openshift-dns/dns-default-mnl8f","openshift-image-registry/node-ca-tn9mk","openshift-monitoring/node-exporter-dn5kk","kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w","openshift-cluster-node-tuning-operator/tuned-nsxh7","openshift-monitoring/alertmanager-main-0","openshift-ovn-kubernetes/ovnkube-node-fk4wd"]
Oct 29 23:13:41.432811 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.432781    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.435943 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.435918    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w5p5k\""
Oct 29 23:13:41.436077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.435953    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-96xtz\""
Oct 29 23:13:41.436077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.435962    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vsvpd\""
Oct 29 23:13:41.436077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.436061    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 29 23:13:41.436077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.436065    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-5wvkd\""
Oct 29 23:13:41.436283 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.436119    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 29 23:13:41.436283 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.436205    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-m6t5c\""
Oct 29 23:13:41.447713 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.447689    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.447811 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.447788    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 29 23:13:41.447865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.447812    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.447865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.447841    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.447865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.447862    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 29 23:13:41.448649 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.448637    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sw5tb\""
Oct 29 23:13:41.448714 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.448704    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 29 23:13:41.455020 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.455005    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 29 23:13:41.455115 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.455095    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 29 23:13:41.455434 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.455421    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wscf4\""
Oct 29 23:13:41.455499 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.455442    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7fjkb\""
Oct 29 23:13:41.455543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.455493    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h2str\""
Oct 29 23:13:41.455574 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.455554    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-z28mk\""
Oct 29 23:13:41.455611 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.455570    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 29 23:13:41.461104 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.461090    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cxqm7\""
Oct 29 23:13:41.461149 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.461132    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zfk9z\""
Oct 29 23:13:41.461672 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.461656    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 29 23:13:41.461766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.461661    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-88spp\""
Oct 29 23:13:41.461766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.461692    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 29 23:13:41.461766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.461755    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-rlkhk\""
Oct 29 23:13:41.461891 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.461778    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8hh8s\""
Oct 29 23:13:41.462019 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462003    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 29 23:13:41.462062 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462021    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 29 23:13:41.462062 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462040    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 29 23:13:41.462148 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462064    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 29 23:13:41.462148 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462087    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 29 23:13:41.462148 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462091    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 29 23:13:41.462148 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462098    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 29 23:13:41.462148 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462098    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 29 23:13:41.462148 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462113    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 29 23:13:41.462148 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462142    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 29 23:13:41.462148 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462148    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 29 23:13:41.462417 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462164    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 29 23:13:41.462417 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462178    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 29 23:13:41.462417 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462245    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lpzjf\""
Oct 29 23:13:41.462417 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462247    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fnb2j\""
Oct 29 23:13:41.462417 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462277    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 29 23:13:41.462566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462426    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-cwxm6\""
Oct 29 23:13:41.462566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462439    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 29 23:13:41.462566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462497    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 29 23:13:41.462566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462500    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4nk8m\""
Oct 29 23:13:41.462680 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462575    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 29 23:13:41.462680 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462591    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pk5sf\""
Oct 29 23:13:41.462761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462686    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kcr8p\""
Oct 29 23:13:41.462831 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462813    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 29 23:13:41.462932 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.462916    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.468400 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.468379    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 29 23:13:41.468668 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.468638    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 29 23:13:41.469361 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.469338    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.469443 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.469376    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.469535 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.469519    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.470502 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.470474    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 29 23:13:41.472449 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.472431    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.472536 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.472512    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 29 23:13:41.472690 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.472677    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.472761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.472729    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 29 23:13:41.472848 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.472835    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.472976 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.472962    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.473146 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473133    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.473214 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473192    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 29 23:13:41.473265 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473237    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 29 23:13:41.473333 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473318    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.473462 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473444    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.473708 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473638    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.473853 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473785    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 29 23:13:41.473929 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473860    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 29 23:13:41.473929 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473873    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.474031 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473978    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.474031 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473987    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.474031 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473996    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 29 23:13:41.474031 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.473993    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.474272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.474143    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.474272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.474182    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 29 23:13:41.474272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.474202    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 29 23:13:41.474491 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.474286    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.474491 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.474408    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 29 23:13:41.474491 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.474411    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 29 23:13:41.474692 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.474599    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 29 23:13:41.474761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.474696    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.474809 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.474766    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 29 23:13:41.474904 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.474883    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 29 23:13:41.475046 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.475012    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 29 23:13:41.476062 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.476041    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 29 23:13:41.476062 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.476055    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 29 23:13:41.476196 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.476099    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 29 23:13:41.478207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.478190    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4d7v87a6le8jl\""
Oct 29 23:13:41.478280 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.478211    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 29 23:13:41.478345 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.478338    6040 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:41.478480 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.478465    6040 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:41.486203 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.486184    6040 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 29 23:13:41.522684 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.522659    6040 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:13:41.522806 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:41.522714    6040 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:41.535831 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.535811    6040 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:13:41.535917 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:41.535858    6040 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-189.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-189.ec2.internal"
Oct 29 23:13:41.542913 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.542887    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.543022 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.542920    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-volume\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.543022 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.542943    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.543022 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.542968    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-systemd\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.543022 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543002    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-bin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.543022 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543023    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-conf-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.543259 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543077    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeeba32d-22c4-4173-83f0-5b4c391644b0-tmp-dir\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:13:41.543259 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543121    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-system-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.543259 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543165    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c5z9\" (UniqueName: \"kubernetes.io/projected/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-kube-api-access-8c5z9\") pod \"ingress-canary-qlpqj\" (UID: \"9eb8b8c3-aae3-4116-9d9e-244d34ba0271\") " pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:41.543259 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543191    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeeba32d-22c4-4173-83f0-5b4c391644b0-tmp-dir\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:13:41.543259 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543190    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-client-tls\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.543259 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543254    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnw2k\" (UniqueName: \"kubernetes.io/projected/ed669269-4708-416f-8b97-9ee2cc5bf891-kube-api-access-xnw2k\") pod \"downloads-6dd789c8b7-9jhw5\" (UID: \"ed669269-4708-416f-8b97-9ee2cc5bf891\") " pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:41.543499 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543295    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-etc-kubernetes\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.543499 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543311    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-volume\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.543499 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543321    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/916781d5-af66-4d17-ba3f-748bab236eb1-host\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:13:41.543499 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543450    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-lib-modules\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.543499 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543484    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-script-lib\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.543677 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543511    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:41.543677 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543534    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-var-lib-kubelet\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.543677 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543559    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-host-slash\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:13:41.543677 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543576    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-client-tls\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.543903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543695    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2wfh\" (UniqueName: \"kubernetes.io/projected/54081aac-51a7-47aa-8dab-a04c8d68ab5f-kube-api-access-b2wfh\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.543903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543702    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-script-lib\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.543903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543730    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e881171-a183-4788-89d8-4da43869fc3c-tmp\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.543903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543771    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.543903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543790    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-systemd-units\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.543903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543846    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-out\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.543903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543875    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-original-pull-secret\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:41.543903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543891    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543916    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-serving-certs-ca-bundle\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543926    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-out\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.543942    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544023    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwqjz\" (UniqueName: \"kubernetes.io/projected/802fe00a-6d8f-483e-9fb5-2ea266688ad1-kube-api-access-gwqjz\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544031    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544050    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-federate-client-tls\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544067    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-iptables-alerter-script\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544085    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-textfile\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544063    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-serving-certs-ca-bundle\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544140    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544149    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-textfile\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544199    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544234    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/802fe00a-6d8f-483e-9fb5-2ea266688ad1-config-volume\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:41.544247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544239    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-federate-client-tls\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544259    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-host\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544258    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-iptables-alerter-script\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544284    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfnrq\" (UniqueName: \"kubernetes.io/projected/3f949f77-5aa6-4443-a09b-6fed361a0d43-kube-api-access-rfnrq\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544303    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-etc-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544330    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-cert\") pod \"ingress-canary-qlpqj\" (UID: \"9eb8b8c3-aae3-4116-9d9e-244d34ba0271\") " pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544404    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544413    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-client-ca-bundle\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544440    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/802fe00a-6d8f-483e-9fb5-2ea266688ad1-config-volume\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544443    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-registration-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544481    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk7h7\" (UniqueName: \"kubernetes.io/projected/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-kube-api-access-qk7h7\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544518    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544524    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-cert\") pod \"ingress-canary-qlpqj\" (UID: \"9eb8b8c3-aae3-4116-9d9e-244d34ba0271\") " pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544547    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-kubelet-config\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544572    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544601    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-client-ca-bundle\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544616    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544650    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-kubernetes\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544667    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-config\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544686    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-root\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.544711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544683    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544727    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxlfh\" (UniqueName: \"kubernetes.io/projected/22013da1-6ec7-4b6d-95f8-298e97fb1372-kube-api-access-mxlfh\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544791    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysconfig\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544845    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544884    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-agent-certs\") pod \"konnectivity-agent-5z4zk\" (UID: \"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e\") " pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544899    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-env-overrides\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544900    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovnkube-config\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544945    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-tls\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.544977    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545023    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87b4272a-c0a8-4ff8-8b80-cf9d76552032-env-overrides\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545035    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545070    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545120    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-socket-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545152    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/802fe00a-6d8f-483e-9fb5-2ea266688ad1-metrics-tls\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545156    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-tls\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545157    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545171    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-agent-certs\") pod \"konnectivity-agent-5z4zk\" (UID: \"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e\") " pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545190    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-cnibin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545226    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-hostroot\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545251    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545284    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-serving-cert\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.545311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545310    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-k8s-cni-cncf-io\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545327    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-kubelet-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545309    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545334    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545351    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4ef6497-03ed-4ed8-beb9-1cde1f435655-data-volume\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545370    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfhnw\" (UniqueName: \"kubernetes.io/projected/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-api-access-tfhnw\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545426    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-modprobe-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545448    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4ef6497-03ed-4ed8-beb9-1cde1f435655-data-volume\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545501    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-slash\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545523    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/802fe00a-6d8f-483e-9fb5-2ea266688ad1-metrics-tls\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545527    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-systemd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545564    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545586    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-sys\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545613    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-tuned\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545628    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-ovn\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545677    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545679    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-tuned\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545710    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpjpd\" (UniqueName: \"kubernetes.io/projected/2bc4c683-845b-40f3-ac2e-2c785b221257-kube-api-access-vpjpd\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545755    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-cni-binary-copy\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545843    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-etc-selinux\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.545871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545860    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovn-node-metrics-cert\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545899    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-tls\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545928    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545944    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-device-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.545999    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-bin\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546042    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546063    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-daemon-config\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546080    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87b4272a-c0a8-4ff8-8b80-cf9d76552032-ovn-node-metrics-cert\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546089    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-cni-binary-copy\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546097    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4ef6497-03ed-4ed8-beb9-1cde1f435655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546165    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-tls\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546187    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-metrics-server-audit-profiles\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546198    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-daemon-config\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546213    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546302    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546335    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-multus-certs\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546394    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-socket-dir-parent\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546419    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4ef6497-03ed-4ed8-beb9-1cde1f435655-crio-socket\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546446    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qf9c\" (UniqueName: \"kubernetes.io/projected/cbf0e826-b477-47a2-9c6d-08ac105aedd0-kube-api-access-2qf9c\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546513    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/259771b7-e14a-496a-bba1-a8f7364e6c1f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546520    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546515    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cbf0e826-b477-47a2-9c6d-08ac105aedd0-metrics-server-audit-profiles\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.546543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546303    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4ef6497-03ed-4ed8-beb9-1cde1f435655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546559    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lb7c\" (UniqueName: \"kubernetes.io/projected/916781d5-af66-4d17-ba3f-748bab236eb1-kube-api-access-9lb7c\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546758    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-netns\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546777    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-metrics-client-ca\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546792    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-client-certs\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546808    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/802fe00a-6d8f-483e-9fb5-2ea266688ad1-tmp-dir\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546839    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546860    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546875    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/802fe00a-6d8f-483e-9fb5-2ea266688ad1-tmp-dir\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.546990    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdgp9\" (UniqueName: \"kubernetes.io/projected/259771b7-e14a-496a-bba1-a8f7364e6c1f-kube-api-access-gdgp9\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547029    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-metrics-client-ca\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547063    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-console-config\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547143    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cbf0e826-b477-47a2-9c6d-08ac105aedd0-secret-metrics-server-client-certs\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547153    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-tmp\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547146    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547206    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-web-config\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547240    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22013da1-6ec7-4b6d-95f8-298e97fb1372-tmp\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547304    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-oauth-config\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.547319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547329    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547346    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffjqr\" (UniqueName: \"kubernetes.io/projected/87b4272a-c0a8-4ff8-8b80-cf9d76552032-kube-api-access-ffjqr\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547365    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cbf0e826-b477-47a2-9c6d-08ac105aedd0-audit-log\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547409    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-web-config\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547428    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cbf0e826-b477-47a2-9c6d-08ac105aedd0-audit-log\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547478    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed669269-4708-416f-8b97-9ee2cc5bf891-tmp\") pod \"downloads-6dd789c8b7-9jhw5\" (UID: \"ed669269-4708-416f-8b97-9ee2cc5bf891\") " pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547497    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547521    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcxgr\" (UniqueName: \"kubernetes.io/projected/62ec6811-5291-4963-b8f1-52a7ffdc50bc-kube-api-access-gcxgr\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547541    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547558    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-multus\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547568    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed669269-4708-416f-8b97-9ee2cc5bf891-tmp\") pod \"downloads-6dd789c8b7-9jhw5\" (UID: \"ed669269-4708-416f-8b97-9ee2cc5bf891\") " pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547584    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-netd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547591    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62ec6811-5291-4963-b8f1-52a7ffdc50bc-metrics-certs\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547640    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-metrics-client-ca\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547678    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-service-ca\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547756    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547767    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-run\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547836    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-konnectivity-ca\") pod \"konnectivity-agent-5z4zk\" (UID: \"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e\") " pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547864    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlpjn\" (UniqueName: \"kubernetes.io/projected/59d798e8-802a-4a6a-8f5a-063d69972c13-kube-api-access-vlpjn\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547870    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/259771b7-e14a-496a-bba1-a8f7364e6c1f-metrics-client-ca\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547883    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-conf\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547901    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpcwp\" (UniqueName: \"kubernetes.io/projected/eeeba32d-22c4-4173-83f0-5b4c391644b0-kube-api-access-jpcwp\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547916    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-log-socket\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547931    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-system-cni-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.547936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547946    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eeeba32d-22c4-4173-83f0-5b4c391644b0-hosts-file\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547960    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-netns\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547975    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-sys-fs\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547989    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9271cfd4-96c6-46a0-84c9-dcdcdccaae4e-konnectivity-ca\") pod \"konnectivity-agent-5z4zk\" (UID: \"9271cfd4-96c6-46a0-84c9-dcdcdccaae4e\") " pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.547989    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-var-lib-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548045    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548086    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548101    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/916781d5-af66-4d17-ba3f-748bab236eb1-serviceca\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548122    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-kubelet\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548136    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-dbus\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548169    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbng5\" (UniqueName: \"kubernetes.io/projected/0e881171-a183-4788-89d8-4da43869fc3c-kube-api-access-dbng5\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548184    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cnibin\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548224    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548241    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-kubelet\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548256    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-accelerators-collector-config\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548282    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwpxg\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-kube-api-access-pwpxg\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548460    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.548639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548589    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.549113 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548751    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-accelerators-collector-config\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.549113 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548906    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.549113 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548934    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-oauth-serving-cert\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.549113 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548949    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/916781d5-af66-4d17-ba3f-748bab236eb1-serviceca\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:13:41.549113 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548962    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-sys\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.549113 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.548979    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-node-log\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.549113 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.549009    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:41.549113 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.549042    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-wtmp\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.549318 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.549151    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:41.582048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.582024    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpcwp\" (UniqueName: \"kubernetes.io/projected/eeeba32d-22c4-4173-83f0-5b4c391644b0-kube-api-access-jpcwp\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:13:41.583277 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.583259    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffjqr\" (UniqueName: \"kubernetes.io/projected/87b4272a-c0a8-4ff8-8b80-cf9d76552032-kube-api-access-ffjqr\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.583356 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.583338    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lb7c\" (UniqueName: \"kubernetes.io/projected/916781d5-af66-4d17-ba3f-748bab236eb1-kube-api-access-9lb7c\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:13:41.583698 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.583683    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfhnw\" (UniqueName: \"kubernetes.io/projected/f4ef6497-03ed-4ed8-beb9-1cde1f435655-kube-api-access-tfhnw\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:41.584095 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.584079    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwpxg\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-kube-api-access-pwpxg\") pod \"alertmanager-main-0\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:41.586023 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.586008    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxlfh\" (UniqueName: \"kubernetes.io/projected/22013da1-6ec7-4b6d-95f8-298e97fb1372-kube-api-access-mxlfh\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.588200 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.588185    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlpjn\" (UniqueName: \"kubernetes.io/projected/59d798e8-802a-4a6a-8f5a-063d69972c13-kube-api-access-vlpjn\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.589022 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.589007    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcxgr\" (UniqueName: \"kubernetes.io/projected/62ec6811-5291-4963-b8f1-52a7ffdc50bc-kube-api-access-gcxgr\") pod \"network-metrics-daemon-bv5tv\" (UID: \"62ec6811-5291-4963-b8f1-52a7ffdc50bc\") " pod="openshift-multus/network-metrics-daemon-bv5tv"
Oct 29 23:13:41.589199 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.589179    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfnrq\" (UniqueName: \"kubernetes.io/projected/3f949f77-5aa6-4443-a09b-6fed361a0d43-kube-api-access-rfnrq\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.589499 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.589484    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c5z9\" (UniqueName: \"kubernetes.io/projected/9eb8b8c3-aae3-4116-9d9e-244d34ba0271-kube-api-access-8c5z9\") pod \"ingress-canary-qlpqj\" (UID: \"9eb8b8c3-aae3-4116-9d9e-244d34ba0271\") " pod="openshift-ingress-canary/ingress-canary-qlpqj"
Oct 29 23:13:41.589729 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.589713    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwqjz\" (UniqueName: \"kubernetes.io/projected/802fe00a-6d8f-483e-9fb5-2ea266688ad1-kube-api-access-gwqjz\") pod \"dns-default-mnl8f\" (UID: \"802fe00a-6d8f-483e-9fb5-2ea266688ad1\") " pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:41.596886 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.596870    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbssw\" (UniqueName: \"kubernetes.io/projected/fd1c395e-bc72-4038-80cc-4ab131d96fdb-kube-api-access-jbssw\") pod \"network-check-target-xhlf6\" (UID: \"fd1c395e-bc72-4038-80cc-4ab131d96fdb\") " pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:41.606713 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.606690    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpjpd\" (UniqueName: \"kubernetes.io/projected/2bc4c683-845b-40f3-ac2e-2c785b221257-kube-api-access-vpjpd\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.613373 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.613357    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnw2k\" (UniqueName: \"kubernetes.io/projected/ed669269-4708-416f-8b97-9ee2cc5bf891-kube-api-access-xnw2k\") pod \"downloads-6dd789c8b7-9jhw5\" (UID: \"ed669269-4708-416f-8b97-9ee2cc5bf891\") " pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:41.617349 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.617309    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk7h7\" (UniqueName: \"kubernetes.io/projected/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-kube-api-access-qk7h7\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:13:41.619771 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.619728    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qf9c\" (UniqueName: \"kubernetes.io/projected/cbf0e826-b477-47a2-9c6d-08ac105aedd0-kube-api-access-2qf9c\") pod \"metrics-server-78f9fd9c95-fj6xw\" (UID: \"cbf0e826-b477-47a2-9c6d-08ac105aedd0\") " pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:41.635897 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.635877    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2wfh\" (UniqueName: \"kubernetes.io/projected/54081aac-51a7-47aa-8dab-a04c8d68ab5f-kube-api-access-b2wfh\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.649849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649825    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-system-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.649914 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649854    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-etc-kubernetes\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.649914 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649870    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/916781d5-af66-4d17-ba3f-748bab236eb1-host\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:13:41.649914 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649884    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-lib-modules\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.649914 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649900    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-var-lib-kubelet\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649921    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-etc-kubernetes\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649922    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-host-slash\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649943    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/916781d5-af66-4d17-ba3f-748bab236eb1-host\") pod \"node-ca-tn9mk\" (UID: \"916781d5-af66-4d17-ba3f-748bab236eb1\") " pod="openshift-image-registry/node-ca-tn9mk"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649959    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5-host-slash\") pod \"iptables-alerter-vk7hb\" (UID: \"6bd1e8d6-581b-4e23-89bd-6a9e5854ffa5\") " pod="openshift-network-operator/iptables-alerter-vk7hb"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649965    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e881171-a183-4788-89d8-4da43869fc3c-tmp\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649972    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-system-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.649996    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-systemd-units\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650018    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-host\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650021    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-var-lib-kubelet\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650033    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-etc-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650049    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-lib-modules\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650065    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-systemd-units\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650066    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-registration-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.650074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650076    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-host\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650092    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-etc-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650098    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-kubelet-config\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650127    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-kubernetes\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650152    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-root\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650158    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-registration-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650169    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-kubelet-config\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650186    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-kubernetes\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650177    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysconfig\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650205    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-root\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650226    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-socket-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650235    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysconfig\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650254    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-cnibin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650277    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-hostroot\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650286    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-socket-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650301    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e881171-a183-4788-89d8-4da43869fc3c-tmp\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650304    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650325    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-hostroot\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650330    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-serving-cert\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650353    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650355    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-k8s-cni-cncf-io\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650334    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-cnibin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650384    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-kubelet-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650428    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-modprobe-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650387    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-k8s-cni-cncf-io\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650433    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-kubelet-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650456    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-slash\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650478    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-systemd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650502    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650529    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-sys\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650536    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-slash\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650540    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-systemd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650529    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-modprobe-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650558    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650552    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-ovn\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650578    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-sys\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650584    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-run-ovn\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650596    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650613    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-etc-selinux\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650627    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650632    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-ovn-kubernetes\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650640    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-device-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650655    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-bin\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650648    6040 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650676    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-multus-certs\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650691    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-socket-dir-parent\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650698    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-device-dir\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650708    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4ef6497-03ed-4ed8-beb9-1cde1f435655-crio-socket\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650725    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-netns\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650726    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-d\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650764    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-console-config\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650763    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-etc-selinux\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.650722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650779    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-bin\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650789    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-multus-certs\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650789    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-socket-dir-parent\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650792    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-run-netns\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650816    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4ef6497-03ed-4ed8-beb9-1cde1f435655-crio-socket\") pod \"insights-runtime-extractor-fk7ml\" (UID: \"f4ef6497-03ed-4ed8-beb9-1cde1f435655\") " pod="openshift-insights/insights-runtime-extractor-fk7ml"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650863    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-oauth-config\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650962    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650978    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-multus\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.650992    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-netd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651028    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-service-ca\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651035    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-multus\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651038    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-cni-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651052    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-run\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651076    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-conf\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651092    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-cni-netd\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651101    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-log-socket\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651139    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-log-socket\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651175    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-run\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651227    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-sysctl-conf\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651215    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-system-cni-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651272    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eeeba32d-22c4-4173-83f0-5b4c391644b0-hosts-file\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651273    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-system-cni-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651321    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eeeba32d-22c4-4173-83f0-5b4c391644b0-hosts-file\") pod \"node-resolver-n4lkv\" (UID: \"eeeba32d-22c4-4173-83f0-5b4c391644b0\") " pod="openshift-dns/node-resolver-n4lkv"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651325    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-netns\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651352    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-sys-fs\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651362    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-run-netns\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651393    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-var-lib-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651421    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-kubelet\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651436    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-var-lib-openvswitch\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651441    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f949f77-5aa6-4443-a09b-6fed361a0d43-sys-fs\") pod \"aws-ebs-csi-driver-node-db85w\" (UID: \"3f949f77-5aa6-4443-a09b-6fed361a0d43\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db85w"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651444    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-dbus\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651490    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbng5\" (UniqueName: \"kubernetes.io/projected/0e881171-a183-4788-89d8-4da43869fc3c-kube-api-access-dbng5\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651468    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-console-config\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651506    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8ac2ba2-0d4b-4498-a7c6-658412d60ccf-dbus\") pod \"global-pull-secret-syncer-wht7s\" (UID: \"a8ac2ba2-0d4b-4498-a7c6-658412d60ccf\") " pod="kube-system/global-pull-secret-syncer-wht7s"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651503    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-host-kubelet\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651518    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cnibin\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651554    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-kubelet\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651558    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-cnibin\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651590    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-kubelet\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651595    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-oauth-serving-cert\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651618    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-sys\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651623    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-service-ca\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651641    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-node-log\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651666    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-wtmp\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651676    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-sys\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651695    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651731    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-systemd\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651765    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54081aac-51a7-47aa-8dab-a04c8d68ab5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wmfk5\" (UID: \"54081aac-51a7-47aa-8dab-a04c8d68ab5f\") " pod="openshift-multus/multus-additional-cni-plugins-wmfk5"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651785    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-bin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651811    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-conf-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651852    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-host-var-lib-cni-bin\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651869    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2bc4c683-845b-40f3-ac2e-2c785b221257-multus-conf-dir\") pod \"multus-95nsq\" (UID: \"2bc4c683-845b-40f3-ac2e-2c785b221257\") " pod="openshift-multus/multus-95nsq"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651872    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22013da1-6ec7-4b6d-95f8-298e97fb1372-etc-systemd\") pod \"tuned-nsxh7\" (UID: \"22013da1-6ec7-4b6d-95f8-298e97fb1372\") " pod="openshift-cluster-node-tuning-operator/tuned-nsxh7"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651929    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/59d798e8-802a-4a6a-8f5a-063d69972c13-node-exporter-wtmp\") pod \"node-exporter-dn5kk\" (UID: \"59d798e8-802a-4a6a-8f5a-063d69972c13\") " pod="openshift-monitoring/node-exporter-dn5kk"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.651976    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b4272a-c0a8-4ff8-8b80-cf9d76552032-node-log\") pod \"ovnkube-node-fk4wd\" (UID: \"87b4272a-c0a8-4ff8-8b80-cf9d76552032\") " pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:41.652584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.652194    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-oauth-serving-cert\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.654306 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.653139    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-serving-cert\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.654306 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.653190    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-oauth-config\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.664905 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.664887    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdgp9\" (UniqueName: \"kubernetes.io/projected/259771b7-e14a-496a-bba1-a8f7364e6c1f-kube-api-access-gdgp9\") pod \"telemeter-client-79699f4d76-w6hb5\" (UID: \"259771b7-e14a-496a-bba1-a8f7364e6c1f\") " pod="openshift-monitoring/telemeter-client-79699f4d76-w6hb5"
Oct 29 23:13:41.693652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.693626    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbng5\" (UniqueName: \"kubernetes.io/projected/0e881171-a183-4788-89d8-4da43869fc3c-kube-api-access-dbng5\") pod \"console-7c654d988-5jnqt\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") " pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:41.730621 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.730602    6040 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 29 23:13:41.744429 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:41.744405    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:42.050537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:42.050502    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c654d988-5jnqt"]
Oct 29 23:13:42.054953 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:42.054928    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e881171_a183_4788_89d8_4da43869fc3c.slice/crio-3dad951b6217976c1af259c4b0a21277c8c4809d41bc71ba12a13b8baa30d021 WatchSource:0}: Error finding container 3dad951b6217976c1af259c4b0a21277c8c4809d41bc71ba12a13b8baa30d021: Status 404 returned error can't find the container with id 3dad951b6217976c1af259c4b0a21277c8c4809d41bc71ba12a13b8baa30d021
Oct 29 23:13:42.483932 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:42.483893    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c654d988-5jnqt" event={"ID":"0e881171-a183-4788-89d8-4da43869fc3c","Type":"ContainerStarted","Data":"3dad951b6217976c1af259c4b0a21277c8c4809d41bc71ba12a13b8baa30d021"}
Oct 29 23:13:44.481968 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:44.481932    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mnl8f"
Oct 29 23:13:44.757698 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:44.757625    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:45.496244 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:45.496200    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c654d988-5jnqt" event={"ID":"0e881171-a183-4788-89d8-4da43869fc3c","Type":"ContainerStarted","Data":"f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42"}
Oct 29 23:13:45.544608 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:45.544124    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7c654d988-5jnqt" podStartSLOduration=2.726957822 podStartE2EDuration="5.544110407s" totalImagesPullingTime="2.817152585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:45.543402102 +0000 UTC m=+5.238510982" watchObservedRunningTime="2026-10-29 23:13:45.544110407 +0000 UTC m=+5.239219280"
Oct 29 23:13:47.141572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:47.141529    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:13:47.344030 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:47.343991    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:47.349463 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:47.349436    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:47.505777 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:47.505680    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:47.509844 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:47.509820    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:13:49.262597 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:49.262562    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:49.262995 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:49.262837    6040 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:13:49.896113 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:49.896083    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fk4wd"
Oct 29 23:13:50.012271 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.012236    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cbd9875f7-kgqz5"]
Oct 29 23:13:50.015585 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.015561    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.067592 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.067543    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 29 23:13:50.068182 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.068161    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cbd9875f7-kgqz5"]
Oct 29 23:13:50.126301 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.126265    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-trusted-ca-bundle\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.126301 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.126302    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-oauth-config\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.126487 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.126324    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-oauth-serving-cert\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.126519 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.126475    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-console-config\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.126565 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.126525    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-service-ca\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.126623 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.126568    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-serving-cert\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.126623 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.126608    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72v46\" (UniqueName: \"kubernetes.io/projected/8ecd186b-7077-4136-82cf-a9804d85052c-kube-api-access-72v46\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.126706 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.126689    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ecd186b-7077-4136-82cf-a9804d85052c-tmp\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.227682 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.227601    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72v46\" (UniqueName: \"kubernetes.io/projected/8ecd186b-7077-4136-82cf-a9804d85052c-kube-api-access-72v46\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.227682 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.227650    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ecd186b-7077-4136-82cf-a9804d85052c-tmp\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.227938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.227706    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-trusted-ca-bundle\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.227938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.227752    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-oauth-config\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.227938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.227779    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-oauth-serving-cert\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.227938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.227812    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-console-config\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.227938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.227844    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-service-ca\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.227938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.227875    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-serving-cert\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.228222 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.228183    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ecd186b-7077-4136-82cf-a9804d85052c-tmp\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.228671 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.228644    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-oauth-serving-cert\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.228784 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.228656    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-service-ca\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.228784 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.228722    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-console-config\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.228919 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.228786    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-trusted-ca-bundle\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.230526 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.230501    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-serving-cert\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.230636 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.230528    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-oauth-config\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.251335 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.251305    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72v46\" (UniqueName: \"kubernetes.io/projected/8ecd186b-7077-4136-82cf-a9804d85052c-kube-api-access-72v46\") pod \"console-7cbd9875f7-kgqz5\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") " pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.329280 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.329242    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:50.633939 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:50.633903    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cbd9875f7-kgqz5"]
Oct 29 23:13:50.637650 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:50.637619    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ecd186b_7077_4136_82cf_a9804d85052c.slice/crio-e12c2fb9562d933f3d3f47ffc184bdebd0c313e0830a51456371199e652f741b WatchSource:0}: Error finding container e12c2fb9562d933f3d3f47ffc184bdebd0c313e0830a51456371199e652f741b: Status 404 returned error can't find the container with id e12c2fb9562d933f3d3f47ffc184bdebd0c313e0830a51456371199e652f741b
Oct 29 23:13:51.523624 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:51.523581    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbd9875f7-kgqz5" event={"ID":"8ecd186b-7077-4136-82cf-a9804d85052c","Type":"ContainerStarted","Data":"2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474"}
Oct 29 23:13:51.524118 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:51.523632    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbd9875f7-kgqz5" event={"ID":"8ecd186b-7077-4136-82cf-a9804d85052c","Type":"ContainerStarted","Data":"e12c2fb9562d933f3d3f47ffc184bdebd0c313e0830a51456371199e652f741b"}
Oct 29 23:13:51.571337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:51.571279    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7cbd9875f7-kgqz5" podStartSLOduration=2.571263373 podStartE2EDuration="2.571263373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:51.570218374 +0000 UTC m=+11.265327277" watchObservedRunningTime="2026-10-29 23:13:51.571263373 +0000 UTC m=+11.266372304"
Oct 29 23:13:52.737636 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:52.737602    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-78f9fd9c95-fj6xw"
Oct 29 23:13:53.519005 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:53.518967    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:53.524312 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:53.524290    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:54.376759 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.376710    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cbd9875f7-kgqz5"]
Oct 29 23:13:54.480685 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.480652    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d74c4fd6f-gwb47"]
Oct 29 23:13:54.505144 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.505108    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.509651 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.509615    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d74c4fd6f-gwb47"]
Oct 29 23:13:54.533660 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.533629    6040 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:13:54.662670 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.662635    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-console-config\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.662963 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.662683    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6df68024-d974-4b31-9858-092da3b9eb9d-console-serving-cert\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.662963 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.662711    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-oauth-serving-cert\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.662963 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.662831    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-trusted-ca-bundle\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.663123 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.662988    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6df68024-d974-4b31-9858-092da3b9eb9d-tmp\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.663175 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.663158    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-service-ca\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.663231 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.663196    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mthhv\" (UniqueName: \"kubernetes.io/projected/6df68024-d974-4b31-9858-092da3b9eb9d-kube-api-access-mthhv\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.663300 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.663279    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6df68024-d974-4b31-9858-092da3b9eb9d-console-oauth-config\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.764568 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.764532    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-service-ca\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.764568 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.764570    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mthhv\" (UniqueName: \"kubernetes.io/projected/6df68024-d974-4b31-9858-092da3b9eb9d-kube-api-access-mthhv\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.764818 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.764623    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6df68024-d974-4b31-9858-092da3b9eb9d-console-oauth-config\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.764818 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.764669    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-console-config\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.764818 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.764698    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6df68024-d974-4b31-9858-092da3b9eb9d-console-serving-cert\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.764818 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.764723    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-oauth-serving-cert\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.764818 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.764768    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-trusted-ca-bundle\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.764818 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.764796    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6df68024-d974-4b31-9858-092da3b9eb9d-tmp\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.765258 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.765228    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6df68024-d974-4b31-9858-092da3b9eb9d-tmp\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.765526 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.765500    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-service-ca\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.765630 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.765524    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-oauth-serving-cert\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.765865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.765819    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-console-config\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.765976 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.765939    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6df68024-d974-4b31-9858-092da3b9eb9d-trusted-ca-bundle\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.767628 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.767605    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6df68024-d974-4b31-9858-092da3b9eb9d-console-oauth-config\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.767725 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.767637    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6df68024-d974-4b31-9858-092da3b9eb9d-console-serving-cert\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.788729 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.788687    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mthhv\" (UniqueName: \"kubernetes.io/projected/6df68024-d974-4b31-9858-092da3b9eb9d-kube-api-access-mthhv\") pod \"console-6d74c4fd6f-gwb47\" (UID: \"6df68024-d974-4b31-9858-092da3b9eb9d\") " pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.829597 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.829563    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:13:54.948857 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:54.948795    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5z4zk"
Oct 29 23:13:55.158956 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:55.158927    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d74c4fd6f-gwb47"]
Oct 29 23:13:55.162628 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:13:55.162591    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df68024_d974_4b31_9858_092da3b9eb9d.slice/crio-f1675a21f19c03380c242f14125d9b0ce7f4f17e621cfcf357e1b63524730b18 WatchSource:0}: Error finding container f1675a21f19c03380c242f14125d9b0ce7f4f17e621cfcf357e1b63524730b18: Status 404 returned error can't find the container with id f1675a21f19c03380c242f14125d9b0ce7f4f17e621cfcf357e1b63524730b18
Oct 29 23:13:55.539016 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:55.538944    6040 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:13:55.539016 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:55.538936    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d74c4fd6f-gwb47" event={"ID":"6df68024-d974-4b31-9858-092da3b9eb9d","Type":"ContainerStarted","Data":"b4be2f147b2fc7fb710caaabc5211a2b50cb86e52c6fe8234c5112431b4a567b"}
Oct 29 23:13:55.539535 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:55.539016    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d74c4fd6f-gwb47" event={"ID":"6df68024-d974-4b31-9858-092da3b9eb9d","Type":"ContainerStarted","Data":"f1675a21f19c03380c242f14125d9b0ce7f4f17e621cfcf357e1b63524730b18"}
Oct 29 23:13:55.635043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:55.634984    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6d74c4fd6f-gwb47" podStartSLOduration=1.634972238 podStartE2EDuration="1.634972238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:55.633965525 +0000 UTC m=+15.329074405" watchObservedRunningTime="2026-10-29 23:13:55.634972238 +0000 UTC m=+15.330081107"
Oct 29 23:13:57.034404 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:57.034370    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:57.038777 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:57.038752    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:13:57.549270 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:57.549238    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-9jhw5" event={"ID":"ed669269-4708-416f-8b97-9ee2cc5bf891","Type":"ContainerStarted","Data":"e86760981c185ccd85a2bb5b3a34fa4597cf15b34939720ea33fbb711aa58ced"}
Oct 29 23:13:57.549589 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:57.549532    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:57.559251 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:57.559218    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-9jhw5"
Oct 29 23:13:58.433333 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.433296    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 29 23:13:58.433817 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.433636    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="alertmanager" containerID="cri-o://65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce" gracePeriod=120
Oct 29 23:13:58.434042 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.433977    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy" containerID="cri-o://8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7" gracePeriod=120
Oct 29 23:13:58.434042 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.434019    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy-web" containerID="cri-o://eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7" gracePeriod=120
Oct 29 23:13:58.434226 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.434046    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="config-reloader" containerID="cri-o://29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b" gracePeriod=120
Oct 29 23:13:58.434226 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.433993    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="prom-label-proxy" containerID="cri-o://9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb" gracePeriod=120
Oct 29 23:13:58.434226 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.434136    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy-metric" containerID="cri-o://e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03" gracePeriod=120
Oct 29 23:13:58.566022 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.565990    6040 generic.go:356] "Generic (PLEG): container finished" podID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerID="9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb" exitCode=0
Oct 29 23:13:58.566190 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.566066    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerDied","Data":"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"}
Oct 29 23:13:58.773122 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.773082    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:58.795730 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.795693    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-web\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.795941 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.795771    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-tls-assets\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-tls-assets\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.795941 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.795817    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.795941 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.795844    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-metrics-client-ca\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.795941 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.795873    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-trusted-ca-bundle\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.795941 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.795916    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-kube-api-access-pwpxg\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-kube-api-access-pwpxg\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.796213 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.795967    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-cluster-tls-config\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.796213 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.796006    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-main-tls\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.796213 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.796034    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-out\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.796213 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.796069    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-main-db\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.796213 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.796095    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-volume\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-volume\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.796213 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.796127    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-web-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-web-config\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.796213 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.796165    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy\") pod \"cf091e44-0c38-428f-a14f-f3a44f9b178b\" (UID: \"cf091e44-0c38-428f-a14f-f3a44f9b178b\") "
Oct 29 23:13:58.796566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.796522    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-metrics-client-ca" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:13:58.796566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.796531    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-trusted-ca-bundle" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:13:58.797112 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.796783    6040 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-metrics-client-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.797112 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.796810    6040 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.797112 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.796910    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-main-db" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:13:58.801639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.801609    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-metric" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:13:58.802012 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.801983    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-tls-assets" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:13:58.803497 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.803468    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-main-tls" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:13:58.803600 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.803484    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-kube-api-access-pwpxg" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "kube-api-access-pwpxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:13:58.803788 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.803729    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-web" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:13:58.803910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.803885    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:13:58.804017 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.803825    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-volume" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:13:58.804182 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.804162    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-out" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:13:58.806555 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.806530    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-cluster-tls-config" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:13:58.813314 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.813288    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-web-config" pod "cf091e44-0c38-428f-a14f-f3a44f9b178b" (UID: "cf091e44-0c38-428f-a14f-f3a44f9b178b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:13:58.898197 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.898152    6040 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.898197 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.898195    6040 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-tls-assets\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.898420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.898211    6040 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.898420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.898227    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwpxg\" (UniqueName: \"kubernetes.io/projected/cf091e44-0c38-428f-a14f-f3a44f9b178b-kube-api-access-pwpxg\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.898420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.898244    6040 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-cluster-tls-config\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.898420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.898258    6040 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-main-tls\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.898420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.898270    6040 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-out\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.898420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.898283    6040 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cf091e44-0c38-428f-a14f-f3a44f9b178b-alertmanager-main-db\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.898420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.898295    6040 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-config-volume\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.898420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.898310    6040 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-web-config\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.898420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:58.898323    6040 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf091e44-0c38-428f-a14f-f3a44f9b178b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:13:59.574420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574381    6040 generic.go:356] "Generic (PLEG): container finished" podID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerID="e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03" exitCode=0
Oct 29 23:13:59.574420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574413    6040 generic.go:356] "Generic (PLEG): container finished" podID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerID="8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7" exitCode=0
Oct 29 23:13:59.574420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574420    6040 generic.go:356] "Generic (PLEG): container finished" podID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerID="eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7" exitCode=0
Oct 29 23:13:59.574420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574427    6040 generic.go:356] "Generic (PLEG): container finished" podID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerID="29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b" exitCode=0
Oct 29 23:13:59.575044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574432    6040 generic.go:356] "Generic (PLEG): container finished" podID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerID="65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce" exitCode=0
Oct 29 23:13:59.575044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574465    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerDied","Data":"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"}
Oct 29 23:13:59.575044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574482    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.575044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574540    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerDied","Data":"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"}
Oct 29 23:13:59.575044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574575    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerDied","Data":"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"}
Oct 29 23:13:59.575044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574599    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerDied","Data":"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"}
Oct 29 23:13:59.575044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574624    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerDied","Data":"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"}
Oct 29 23:13:59.575044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574654    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cf091e44-0c38-428f-a14f-f3a44f9b178b","Type":"ContainerDied","Data":"bbcdc18091b22a47939271a133581135955337afb04756ce9d6709a4521e29b8"}
Oct 29 23:13:59.575044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.574654    6040 scope.go:122] "RemoveContainer" containerID="9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"
Oct 29 23:13:59.584559 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.584539    6040 scope.go:122] "RemoveContainer" containerID="e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"
Oct 29 23:13:59.592651 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.592633    6040 scope.go:122] "RemoveContainer" containerID="8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"
Oct 29 23:13:59.600497 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.600476    6040 scope.go:122] "RemoveContainer" containerID="eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"
Oct 29 23:13:59.608317 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.608297    6040 scope.go:122] "RemoveContainer" containerID="29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"
Oct 29 23:13:59.616393 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.616373    6040 scope.go:122] "RemoveContainer" containerID="65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"
Oct 29 23:13:59.624604 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.624583    6040 scope.go:122] "RemoveContainer" containerID="0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"
Oct 29 23:13:59.632791 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.632766    6040 scope.go:122] "RemoveContainer" containerID="9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"
Oct 29 23:13:59.633138 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:59.633110    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb\": container with ID starting with 9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb not found: ID does not exist" containerID="9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"
Oct 29 23:13:59.633221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.633149    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"} err="failed to get container status \"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb\": rpc error: code = NotFound desc = could not find container \"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb\": container with ID starting with 9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb not found: ID does not exist"
Oct 29 23:13:59.633221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.633185    6040 scope.go:122] "RemoveContainer" containerID="e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"
Oct 29 23:13:59.633504 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:59.633475    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03\": container with ID starting with e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03 not found: ID does not exist" containerID="e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"
Oct 29 23:13:59.633598 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.633510    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"} err="failed to get container status \"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03\": rpc error: code = NotFound desc = could not find container \"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03\": container with ID starting with e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03 not found: ID does not exist"
Oct 29 23:13:59.633598 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.633531    6040 scope.go:122] "RemoveContainer" containerID="8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"
Oct 29 23:13:59.633841 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:59.633805    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7\": container with ID starting with 8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7 not found: ID does not exist" containerID="8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"
Oct 29 23:13:59.633921 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.633833    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"} err="failed to get container status \"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7\": rpc error: code = NotFound desc = could not find container \"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7\": container with ID starting with 8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7 not found: ID does not exist"
Oct 29 23:13:59.633921 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.633855    6040 scope.go:122] "RemoveContainer" containerID="eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"
Oct 29 23:13:59.634143 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:59.634125    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7\": container with ID starting with eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7 not found: ID does not exist" containerID="eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"
Oct 29 23:13:59.634204 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.634147    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"} err="failed to get container status \"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7\": rpc error: code = NotFound desc = could not find container \"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7\": container with ID starting with eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7 not found: ID does not exist"
Oct 29 23:13:59.634204 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.634182    6040 scope.go:122] "RemoveContainer" containerID="29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"
Oct 29 23:13:59.634494 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:59.634463    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b\": container with ID starting with 29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b not found: ID does not exist" containerID="29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"
Oct 29 23:13:59.634580 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.634497    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"} err="failed to get container status \"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b\": rpc error: code = NotFound desc = could not find container \"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b\": container with ID starting with 29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b not found: ID does not exist"
Oct 29 23:13:59.634580 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.634513    6040 scope.go:122] "RemoveContainer" containerID="65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"
Oct 29 23:13:59.634818 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:59.634797    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce\": container with ID starting with 65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce not found: ID does not exist" containerID="65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"
Oct 29 23:13:59.634928 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.634830    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"} err="failed to get container status \"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce\": rpc error: code = NotFound desc = could not find container \"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce\": container with ID starting with 65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce not found: ID does not exist"
Oct 29 23:13:59.634928 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.634846    6040 scope.go:122] "RemoveContainer" containerID="0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"
Oct 29 23:13:59.635123 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:59.635100    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66\": container with ID starting with 0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66 not found: ID does not exist" containerID="0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"
Oct 29 23:13:59.635176 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.635132    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"} err="failed to get container status \"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66\": rpc error: code = NotFound desc = could not find container \"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66\": container with ID starting with 0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66 not found: ID does not exist"
Oct 29 23:13:59.635176 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.635152    6040 scope.go:122] "RemoveContainer" containerID="9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"
Oct 29 23:13:59.635426 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.635401    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"} err="failed to get container status \"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb\": rpc error: code = NotFound desc = could not find container \"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb\": container with ID starting with 9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb not found: ID does not exist"
Oct 29 23:13:59.635501 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.635427    6040 scope.go:122] "RemoveContainer" containerID="e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"
Oct 29 23:13:59.635694 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.635669    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"} err="failed to get container status \"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03\": rpc error: code = NotFound desc = could not find container \"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03\": container with ID starting with e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03 not found: ID does not exist"
Oct 29 23:13:59.635787 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.635695    6040 scope.go:122] "RemoveContainer" containerID="8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"
Oct 29 23:13:59.636060 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.636037    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"} err="failed to get container status \"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7\": rpc error: code = NotFound desc = could not find container \"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7\": container with ID starting with 8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7 not found: ID does not exist"
Oct 29 23:13:59.636060 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.636058    6040 scope.go:122] "RemoveContainer" containerID="eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"
Oct 29 23:13:59.636384 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.636356    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"} err="failed to get container status \"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7\": rpc error: code = NotFound desc = could not find container \"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7\": container with ID starting with eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7 not found: ID does not exist"
Oct 29 23:13:59.636384 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.636384    6040 scope.go:122] "RemoveContainer" containerID="29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"
Oct 29 23:13:59.636670 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.636650    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"} err="failed to get container status \"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b\": rpc error: code = NotFound desc = could not find container \"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b\": container with ID starting with 29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b not found: ID does not exist"
Oct 29 23:13:59.636800 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.636671    6040 scope.go:122] "RemoveContainer" containerID="65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"
Oct 29 23:13:59.636917 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.636892    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"} err="failed to get container status \"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce\": rpc error: code = NotFound desc = could not find container \"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce\": container with ID starting with 65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce not found: ID does not exist"
Oct 29 23:13:59.636917 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.636914    6040 scope.go:122] "RemoveContainer" containerID="0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"
Oct 29 23:13:59.637152 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.637134    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"} err="failed to get container status \"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66\": rpc error: code = NotFound desc = could not find container \"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66\": container with ID starting with 0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66 not found: ID does not exist"
Oct 29 23:13:59.637152 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.637154    6040 scope.go:122] "RemoveContainer" containerID="9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"
Oct 29 23:13:59.637526 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.637500    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"} err="failed to get container status \"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb\": rpc error: code = NotFound desc = could not find container \"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb\": container with ID starting with 9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb not found: ID does not exist"
Oct 29 23:13:59.637526 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.637524    6040 scope.go:122] "RemoveContainer" containerID="e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"
Oct 29 23:13:59.637794 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.637775    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"} err="failed to get container status \"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03\": rpc error: code = NotFound desc = could not find container \"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03\": container with ID starting with e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03 not found: ID does not exist"
Oct 29 23:13:59.637794 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.637792    6040 scope.go:122] "RemoveContainer" containerID="8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"
Oct 29 23:13:59.638061 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.638041    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"} err="failed to get container status \"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7\": rpc error: code = NotFound desc = could not find container \"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7\": container with ID starting with 8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7 not found: ID does not exist"
Oct 29 23:13:59.638061 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.638059    6040 scope.go:122] "RemoveContainer" containerID="eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"
Oct 29 23:13:59.638361 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.638322    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"} err="failed to get container status \"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7\": rpc error: code = NotFound desc = could not find container \"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7\": container with ID starting with eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7 not found: ID does not exist"
Oct 29 23:13:59.638361 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.638352    6040 scope.go:122] "RemoveContainer" containerID="29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"
Oct 29 23:13:59.638599 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.638576    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"} err="failed to get container status \"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b\": rpc error: code = NotFound desc = could not find container \"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b\": container with ID starting with 29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b not found: ID does not exist"
Oct 29 23:13:59.638672 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.638601    6040 scope.go:122] "RemoveContainer" containerID="65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"
Oct 29 23:13:59.638897 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.638864    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"} err="failed to get container status \"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce\": rpc error: code = NotFound desc = could not find container \"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce\": container with ID starting with 65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce not found: ID does not exist"
Oct 29 23:13:59.638980 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.638906    6040 scope.go:122] "RemoveContainer" containerID="0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"
Oct 29 23:13:59.639187 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.639166    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"} err="failed to get container status \"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66\": rpc error: code = NotFound desc = could not find container \"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66\": container with ID starting with 0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66 not found: ID does not exist"
Oct 29 23:13:59.639267 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.639188    6040 scope.go:122] "RemoveContainer" containerID="9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"
Oct 29 23:13:59.639439 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.639421    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"} err="failed to get container status \"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb\": rpc error: code = NotFound desc = could not find container \"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb\": container with ID starting with 9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb not found: ID does not exist"
Oct 29 23:13:59.639504 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.639440    6040 scope.go:122] "RemoveContainer" containerID="e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"
Oct 29 23:13:59.639717 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.639698    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"} err="failed to get container status \"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03\": rpc error: code = NotFound desc = could not find container \"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03\": container with ID starting with e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03 not found: ID does not exist"
Oct 29 23:13:59.639717 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.639717    6040 scope.go:122] "RemoveContainer" containerID="8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"
Oct 29 23:13:59.640042 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.640021    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"} err="failed to get container status \"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7\": rpc error: code = NotFound desc = could not find container \"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7\": container with ID starting with 8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7 not found: ID does not exist"
Oct 29 23:13:59.640137 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.640042    6040 scope.go:122] "RemoveContainer" containerID="eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"
Oct 29 23:13:59.640354 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.640330    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"} err="failed to get container status \"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7\": rpc error: code = NotFound desc = could not find container \"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7\": container with ID starting with eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7 not found: ID does not exist"
Oct 29 23:13:59.640354 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.640355    6040 scope.go:122] "RemoveContainer" containerID="29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"
Oct 29 23:13:59.640610 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.640588    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"} err="failed to get container status \"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b\": rpc error: code = NotFound desc = could not find container \"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b\": container with ID starting with 29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b not found: ID does not exist"
Oct 29 23:13:59.640610 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.640608    6040 scope.go:122] "RemoveContainer" containerID="65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"
Oct 29 23:13:59.640940 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.640914    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"} err="failed to get container status \"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce\": rpc error: code = NotFound desc = could not find container \"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce\": container with ID starting with 65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce not found: ID does not exist"
Oct 29 23:13:59.640940 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.640941    6040 scope.go:122] "RemoveContainer" containerID="0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"
Oct 29 23:13:59.641234 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.641215    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"} err="failed to get container status \"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66\": rpc error: code = NotFound desc = could not find container \"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66\": container with ID starting with 0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66 not found: ID does not exist"
Oct 29 23:13:59.641310 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.641236    6040 scope.go:122] "RemoveContainer" containerID="9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"
Oct 29 23:13:59.641520 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.641498    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb"} err="failed to get container status \"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb\": rpc error: code = NotFound desc = could not find container \"9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb\": container with ID starting with 9f32325d5b59eac63dc6d0b1a1b72f28dedc0ce18c40d6262344407b0fff7bcb not found: ID does not exist"
Oct 29 23:13:59.641598 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.641520    6040 scope.go:122] "RemoveContainer" containerID="e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"
Oct 29 23:13:59.641821 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.641800    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03"} err="failed to get container status \"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03\": rpc error: code = NotFound desc = could not find container \"e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03\": container with ID starting with e322e47089aeffb2b03486530279d71b6e57d8a817f1daae24e9383345187e03 not found: ID does not exist"
Oct 29 23:13:59.641906 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.641821    6040 scope.go:122] "RemoveContainer" containerID="8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"
Oct 29 23:13:59.642104 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.642081    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7"} err="failed to get container status \"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7\": rpc error: code = NotFound desc = could not find container \"8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7\": container with ID starting with 8266cc870eb42baae045a604e43a721bedbf6462d0fc0d45bfb52b073f004fd7 not found: ID does not exist"
Oct 29 23:13:59.642104 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.642104    6040 scope.go:122] "RemoveContainer" containerID="eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"
Oct 29 23:13:59.642380 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.642358    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7"} err="failed to get container status \"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7\": rpc error: code = NotFound desc = could not find container \"eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7\": container with ID starting with eec520e729d6fa3b6953b95f274e05dc74b116125cd18c1f02ae2ffe742773d7 not found: ID does not exist"
Oct 29 23:13:59.642380 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.642379    6040 scope.go:122] "RemoveContainer" containerID="29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"
Oct 29 23:13:59.642703 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.642679    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b"} err="failed to get container status \"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b\": rpc error: code = NotFound desc = could not find container \"29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b\": container with ID starting with 29ac303755d2295a4dd1929993be277dcd23d47c098046f07f8774ad3910966b not found: ID does not exist"
Oct 29 23:13:59.642798 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.642703    6040 scope.go:122] "RemoveContainer" containerID="65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"
Oct 29 23:13:59.642996 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.642973    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce"} err="failed to get container status \"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce\": rpc error: code = NotFound desc = could not find container \"65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce\": container with ID starting with 65d714ae9890759411dea20611dde64c0fd3d244f19df21703f084830b2f99ce not found: ID does not exist"
Oct 29 23:13:59.643062 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.642998    6040 scope.go:122] "RemoveContainer" containerID="0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"
Oct 29 23:13:59.643313 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.643292    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66"} err="failed to get container status \"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66\": rpc error: code = NotFound desc = could not find container \"0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66\": container with ID starting with 0e3436ec84f71e49c1b8481cf399650ef56721ebad35a1fef9b1420f9b6d4e66 not found: ID does not exist"
Oct 29 23:13:59.646340 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.646319    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 29 23:13:59.658285 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.658259    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 29 23:13:59.718206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718160    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 29 23:13:59.718485 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718466    6040 cpu_manager.go:414] "containerMap: removing container" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="prom-label-proxy"
Oct 29 23:13:59.718547 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718487    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="prom-label-proxy"
Oct 29 23:13:59.718547 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718517    6040 cpu_manager.go:414] "containerMap: removing container" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="alertmanager"
Oct 29 23:13:59.718547 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718525    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="alertmanager"
Oct 29 23:13:59.718547 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718544    6040 cpu_manager.go:414] "containerMap: removing container" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718553    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718563    6040 cpu_manager.go:414] "containerMap: removing container" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="config-reloader"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718570    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="config-reloader"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718577    6040 cpu_manager.go:414] "containerMap: removing container" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="init-config-reloader"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718583    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="init-config-reloader"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718594    6040 cpu_manager.go:414] "containerMap: removing container" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy-metric"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718602    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy-metric"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718612    6040 cpu_manager.go:414] "containerMap: removing container" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy-web"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718619    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy-web"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718678    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718687    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="config-reloader"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718696    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="init-config-reloader"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718705    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy-metric"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718714    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="kube-rbac-proxy-web"
Oct 29 23:13:59.718718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718720    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="prom-label-proxy"
Oct 29 23:13:59.719362 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.718731    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" containerName="alertmanager"
Oct 29 23:13:59.748225 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.748159    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.751895 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.751822    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 29 23:13:59.789665 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.789260    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 29 23:13:59.789665 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.789319    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 29 23:13:59.789665 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.789392    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 29 23:13:59.789665 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.789438    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 29 23:13:59.789665 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.789463    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 29 23:13:59.789665 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.789543    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 29 23:13:59.789665 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.789559    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 29 23:13:59.789665 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.789576    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 29 23:13:59.789665 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.789637    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 29 23:13:59.789665 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.789676    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fnb2j\""
Oct 29 23:13:59.793235 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:13:59.793205    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:13:59.805297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805270    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805424 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805309    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nbzq\" (UniqueName: \"kubernetes.io/projected/1628ae69-8a7c-4ef2-a409-add5a2a379c0-kube-api-access-7nbzq\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805424 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805334    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1628ae69-8a7c-4ef2-a409-add5a2a379c0-config-out\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805523 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805471    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805523 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805503    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805603 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805528    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-config-volume\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805603 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805586    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1628ae69-8a7c-4ef2-a409-add5a2a379c0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805680 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805631    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-web-config\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805715 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805669    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1628ae69-8a7c-4ef2-a409-add5a2a379c0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805715 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805705    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805733    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805797    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1628ae69-8a7c-4ef2-a409-add5a2a379c0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.805936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.805903    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1628ae69-8a7c-4ef2-a409-add5a2a379c0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906282 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906244    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1628ae69-8a7c-4ef2-a409-add5a2a379c0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906455 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906298    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906455 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906349    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nbzq\" (UniqueName: \"kubernetes.io/projected/1628ae69-8a7c-4ef2-a409-add5a2a379c0-kube-api-access-7nbzq\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906455 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906378    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1628ae69-8a7c-4ef2-a409-add5a2a379c0-config-out\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906455 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906443    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906661 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906473    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906661 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906511    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-config-volume\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906661 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906532    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1628ae69-8a7c-4ef2-a409-add5a2a379c0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906661 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906555    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-web-config\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906661 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906580    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1628ae69-8a7c-4ef2-a409-add5a2a379c0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906661 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906605    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906661 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906631    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.906661 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.906656    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1628ae69-8a7c-4ef2-a409-add5a2a379c0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.907492 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.907347    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1628ae69-8a7c-4ef2-a409-add5a2a379c0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.907684 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.907658    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1628ae69-8a7c-4ef2-a409-add5a2a379c0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.908012 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.907825    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1628ae69-8a7c-4ef2-a409-add5a2a379c0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.911254 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.910084    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1628ae69-8a7c-4ef2-a409-add5a2a379c0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.912962 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.912601    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1628ae69-8a7c-4ef2-a409-add5a2a379c0-config-out\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.912962 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.912915    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-web-config\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.915067 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.914547    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.915067 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.914580    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.915067 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.914579    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.915067 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.914983    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-config-volume\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.915067 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.915021    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.915367 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.915168    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1628ae69-8a7c-4ef2-a409-add5a2a379c0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:13:59.930437 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:13:59.930388    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nbzq\" (UniqueName: \"kubernetes.io/projected/1628ae69-8a7c-4ef2-a409-add5a2a379c0-kube-api-access-7nbzq\") pod \"alertmanager-main-0\" (UID: \"1628ae69-8a7c-4ef2-a409-add5a2a379c0\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:14:00.218359 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:00.218320    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 29 23:14:00.423358 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:00.423325    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf091e44-0c38-428f-a14f-f3a44f9b178b" path="/var/lib/kubelet/pods/cf091e44-0c38-428f-a14f-f3a44f9b178b/volumes"
Oct 29 23:14:00.585001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:00.584971    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 29 23:14:00.587283 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:14:00.587249    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1628ae69_8a7c_4ef2_a409_add5a2a379c0.slice/crio-06a22f9ca344f09e52c780139fcd88aa0c51c37e9d55c194ab2757fa48010b00 WatchSource:0}: Error finding container 06a22f9ca344f09e52c780139fcd88aa0c51c37e9d55c194ab2757fa48010b00: Status 404 returned error can't find the container with id 06a22f9ca344f09e52c780139fcd88aa0c51c37e9d55c194ab2757fa48010b00
Oct 29 23:14:01.584610 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:01.584572    6040 generic.go:356] "Generic (PLEG): container finished" podID="1628ae69-8a7c-4ef2-a409-add5a2a379c0" containerID="8ec6732b09a0891c35e73463108253cb5723d168a3c59e629c183c278188d058" exitCode=0
Oct 29 23:14:01.584850 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:01.584620    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1628ae69-8a7c-4ef2-a409-add5a2a379c0","Type":"ContainerDied","Data":"8ec6732b09a0891c35e73463108253cb5723d168a3c59e629c183c278188d058"}
Oct 29 23:14:01.584850 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:01.584675    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1628ae69-8a7c-4ef2-a409-add5a2a379c0","Type":"ContainerStarted","Data":"06a22f9ca344f09e52c780139fcd88aa0c51c37e9d55c194ab2757fa48010b00"}
Oct 29 23:14:02.592909 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:02.592861    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1628ae69-8a7c-4ef2-a409-add5a2a379c0","Type":"ContainerStarted","Data":"fd66538026cdb547ef19a3930c38b99813e9adb3a16f823a329ef2038d7c42f4"}
Oct 29 23:14:02.593462 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:02.592950    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1628ae69-8a7c-4ef2-a409-add5a2a379c0","Type":"ContainerStarted","Data":"de30e0e300ab04c322bffb6d6730b585d932473e7f9c209a0083c362555f0efa"}
Oct 29 23:14:02.593462 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:02.592982    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1628ae69-8a7c-4ef2-a409-add5a2a379c0","Type":"ContainerStarted","Data":"6c66634a921235a33d2f0ee7e39b60d5841fb952bd1c8821de8bf027db66f17a"}
Oct 29 23:14:02.593462 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:02.593004    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1628ae69-8a7c-4ef2-a409-add5a2a379c0","Type":"ContainerStarted","Data":"c52acffb5977e9ffb407e6ea2e6c09ff3c07196e8e103bde1354543d96760ad1"}
Oct 29 23:14:02.593462 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:02.593027    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1628ae69-8a7c-4ef2-a409-add5a2a379c0","Type":"ContainerStarted","Data":"2970c8bb665b760a3336988ec6ae065835c1a5cf20e9853f65ac4a46a944700a"}
Oct 29 23:14:02.593462 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:02.593049    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1628ae69-8a7c-4ef2-a409-add5a2a379c0","Type":"ContainerStarted","Data":"95145ad03a634f0b4553e60e26dc598b55a55eeb7059b2e79a50137652cb8995"}
Oct 29 23:14:02.658467 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:02.658414    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.643058849 podStartE2EDuration="3.65840186s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.343011ms" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:14:02.65770919 +0000 UTC m=+22.352818106" watchObservedRunningTime="2026-10-29 23:14:02.65840186 +0000 UTC m=+22.353510756"
Oct 29 23:14:04.830574 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:04.830532    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:14:04.831201 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:04.830621    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:14:04.836093 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:04.836070    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:14:05.609612 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:05.609580    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d74c4fd6f-gwb47"
Oct 29 23:14:05.687943 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:05.687912    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c654d988-5jnqt"]
Oct 29 23:14:17.144553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:17.144520    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xhlf6"
Oct 29 23:14:20.559143 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.559086    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7cbd9875f7-kgqz5" podUID="8ecd186b-7077-4136-82cf-a9804d85052c" containerName="console" containerID="cri-o://2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474" gracePeriod=15
Oct 29 23:14:20.802198 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.802171    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:14:20.896493 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.896461    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-oauth-config\") pod \"8ecd186b-7077-4136-82cf-a9804d85052c\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") "
Oct 29 23:14:20.896695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.896520    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-trusted-ca-bundle\") pod \"8ecd186b-7077-4136-82cf-a9804d85052c\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") "
Oct 29 23:14:20.896695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.896547    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-console-config\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-console-config\") pod \"8ecd186b-7077-4136-82cf-a9804d85052c\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") "
Oct 29 23:14:20.896695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.896574    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ecd186b-7077-4136-82cf-a9804d85052c-kube-api-access-72v46\" (UniqueName: \"kubernetes.io/projected/8ecd186b-7077-4136-82cf-a9804d85052c-kube-api-access-72v46\") pod \"8ecd186b-7077-4136-82cf-a9804d85052c\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") "
Oct 29 23:14:20.896695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.896606    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-oauth-serving-cert\") pod \"8ecd186b-7077-4136-82cf-a9804d85052c\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") "
Oct 29 23:14:20.896944 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.896724    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-serving-cert\") pod \"8ecd186b-7077-4136-82cf-a9804d85052c\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") "
Oct 29 23:14:20.896944 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.896802    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ecd186b-7077-4136-82cf-a9804d85052c-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ecd186b-7077-4136-82cf-a9804d85052c-tmp\") pod \"8ecd186b-7077-4136-82cf-a9804d85052c\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") "
Oct 29 23:14:20.896944 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.896856    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-service-ca\") pod \"8ecd186b-7077-4136-82cf-a9804d85052c\" (UID: \"8ecd186b-7077-4136-82cf-a9804d85052c\") "
Oct 29 23:14:20.897095 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.897025    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-trusted-ca-bundle" pod "8ecd186b-7077-4136-82cf-a9804d85052c" (UID: "8ecd186b-7077-4136-82cf-a9804d85052c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:14:20.897145 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.897093    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-oauth-serving-cert" pod "8ecd186b-7077-4136-82cf-a9804d85052c" (UID: "8ecd186b-7077-4136-82cf-a9804d85052c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:14:20.897254 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.897106    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-console-config" pod "8ecd186b-7077-4136-82cf-a9804d85052c" (UID: "8ecd186b-7077-4136-82cf-a9804d85052c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:14:20.897338 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.897303    6040 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-oauth-serving-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:20.897338 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.897314    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ecd186b-7077-4136-82cf-a9804d85052c-tmp" pod "8ecd186b-7077-4136-82cf-a9804d85052c" (UID: "8ecd186b-7077-4136-82cf-a9804d85052c"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:14:20.897338 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.897335    6040 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-trusted-ca-bundle\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:20.897472 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.897348    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-service-ca" pod "8ecd186b-7077-4136-82cf-a9804d85052c" (UID: "8ecd186b-7077-4136-82cf-a9804d85052c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:14:20.898908 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.898883    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-serving-cert" pod "8ecd186b-7077-4136-82cf-a9804d85052c" (UID: "8ecd186b-7077-4136-82cf-a9804d85052c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:14:20.898996 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.898930    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ecd186b-7077-4136-82cf-a9804d85052c-kube-api-access-72v46" pod "8ecd186b-7077-4136-82cf-a9804d85052c" (UID: "8ecd186b-7077-4136-82cf-a9804d85052c"). InnerVolumeSpecName "kube-api-access-72v46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:14:20.899035 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.899012    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-oauth-config" pod "8ecd186b-7077-4136-82cf-a9804d85052c" (UID: "8ecd186b-7077-4136-82cf-a9804d85052c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:14:20.998169 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.998133    6040 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-console-config\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:20.998169 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.998162    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72v46\" (UniqueName: \"kubernetes.io/projected/8ecd186b-7077-4136-82cf-a9804d85052c-kube-api-access-72v46\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:20.998169 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.998172    6040 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-serving-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:20.998385 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.998181    6040 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ecd186b-7077-4136-82cf-a9804d85052c-tmp\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:20.998385 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.998191    6040 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ecd186b-7077-4136-82cf-a9804d85052c-service-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:20.998385 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:20.998200    6040 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ecd186b-7077-4136-82cf-a9804d85052c-console-oauth-config\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:21.662570 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:21.662534    6040 generic.go:356] "Generic (PLEG): container finished" podID="8ecd186b-7077-4136-82cf-a9804d85052c" containerID="2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474" exitCode=0
Oct 29 23:14:21.662570 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:21.662568    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbd9875f7-kgqz5" event={"ID":"8ecd186b-7077-4136-82cf-a9804d85052c","Type":"ContainerDied","Data":"2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474"}
Oct 29 23:14:21.663089 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:21.662583    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbd9875f7-kgqz5"
Oct 29 23:14:21.663089 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:21.662600    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbd9875f7-kgqz5" event={"ID":"8ecd186b-7077-4136-82cf-a9804d85052c","Type":"ContainerDied","Data":"e12c2fb9562d933f3d3f47ffc184bdebd0c313e0830a51456371199e652f741b"}
Oct 29 23:14:21.663089 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:21.662619    6040 scope.go:122] "RemoveContainer" containerID="2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474"
Oct 29 23:14:21.670946 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:21.670922    6040 scope.go:122] "RemoveContainer" containerID="2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474"
Oct 29 23:14:21.671214 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:14:21.671197    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474\": container with ID starting with 2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474 not found: ID does not exist" containerID="2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474"
Oct 29 23:14:21.671254 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:21.671223    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474"} err="failed to get container status \"2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474\": rpc error: code = NotFound desc = could not find container \"2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474\": container with ID starting with 2366b7af597a91d970ab37462a83902c1c98900db157d8f77fd3c554651e3474 not found: ID does not exist"
Oct 29 23:14:21.712719 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:21.712688    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cbd9875f7-kgqz5"]
Oct 29 23:14:21.724554 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:21.724528    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cbd9875f7-kgqz5"]
Oct 29 23:14:22.421923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:22.421888    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ecd186b-7077-4136-82cf-a9804d85052c" path="/var/lib/kubelet/pods/8ecd186b-7077-4136-82cf-a9804d85052c/volumes"
Oct 29 23:14:29.778403 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:14:29.778368    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:14:30.708559 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:30.708509    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7c654d988-5jnqt" podUID="0e881171-a183-4788-89d8-4da43869fc3c" containerName="console" containerID="cri-o://f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42" gracePeriod=15
Oct 29 23:14:30.992627 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:30.992556    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:14:31.083542 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.083506    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-oauth-serving-cert\") pod \"0e881171-a183-4788-89d8-4da43869fc3c\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") "
Oct 29 23:14:31.083757 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.083559    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-console-config\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-console-config\") pod \"0e881171-a183-4788-89d8-4da43869fc3c\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") "
Oct 29 23:14:31.083757 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.083594    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-service-ca\") pod \"0e881171-a183-4788-89d8-4da43869fc3c\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") "
Oct 29 23:14:31.083757 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.083624    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e881171-a183-4788-89d8-4da43869fc3c-tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e881171-a183-4788-89d8-4da43869fc3c-tmp\") pod \"0e881171-a183-4788-89d8-4da43869fc3c\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") "
Oct 29 23:14:31.083757 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.083656    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-oauth-config\") pod \"0e881171-a183-4788-89d8-4da43869fc3c\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") "
Oct 29 23:14:31.083987 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.083797    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e881171-a183-4788-89d8-4da43869fc3c-kube-api-access-dbng5\" (UniqueName: \"kubernetes.io/projected/0e881171-a183-4788-89d8-4da43869fc3c-kube-api-access-dbng5\") pod \"0e881171-a183-4788-89d8-4da43869fc3c\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") "
Oct 29 23:14:31.083987 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.083885    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-serving-cert\") pod \"0e881171-a183-4788-89d8-4da43869fc3c\" (UID: \"0e881171-a183-4788-89d8-4da43869fc3c\") "
Oct 29 23:14:31.084092 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.084033    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e881171-a183-4788-89d8-4da43869fc3c-tmp" pod "0e881171-a183-4788-89d8-4da43869fc3c" (UID: "0e881171-a183-4788-89d8-4da43869fc3c"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:14:31.084092 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.084052    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-oauth-serving-cert" pod "0e881171-a183-4788-89d8-4da43869fc3c" (UID: "0e881171-a183-4788-89d8-4da43869fc3c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:14:31.084177 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.084099    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-service-ca" pod "0e881171-a183-4788-89d8-4da43869fc3c" (UID: "0e881171-a183-4788-89d8-4da43869fc3c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:14:31.084177 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.084146    6040 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e881171-a183-4788-89d8-4da43869fc3c-tmp\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:31.084177 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.084168    6040 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-oauth-serving-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:31.084294 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.084278    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-console-config" pod "0e881171-a183-4788-89d8-4da43869fc3c" (UID: "0e881171-a183-4788-89d8-4da43869fc3c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:14:31.085758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.085717    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-oauth-config" pod "0e881171-a183-4788-89d8-4da43869fc3c" (UID: "0e881171-a183-4788-89d8-4da43869fc3c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:14:31.086309 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.086292    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-serving-cert" pod "0e881171-a183-4788-89d8-4da43869fc3c" (UID: "0e881171-a183-4788-89d8-4da43869fc3c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:14:31.086370 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.086324    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e881171-a183-4788-89d8-4da43869fc3c-kube-api-access-dbng5" pod "0e881171-a183-4788-89d8-4da43869fc3c" (UID: "0e881171-a183-4788-89d8-4da43869fc3c"). InnerVolumeSpecName "kube-api-access-dbng5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:14:31.184885 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.184851    6040 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-console-config\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:31.184885 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.184881    6040 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e881171-a183-4788-89d8-4da43869fc3c-service-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:31.184885 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.184890    6040 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-oauth-config\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:31.185100 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.184899    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbng5\" (UniqueName: \"kubernetes.io/projected/0e881171-a183-4788-89d8-4da43869fc3c-kube-api-access-dbng5\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:31.185100 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.184908    6040 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e881171-a183-4788-89d8-4da43869fc3c-console-serving-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:14:31.696094 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.696061    6040 generic.go:356] "Generic (PLEG): container finished" podID="0e881171-a183-4788-89d8-4da43869fc3c" containerID="f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42" exitCode=0
Oct 29 23:14:31.696286 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.696115    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c654d988-5jnqt"
Oct 29 23:14:31.696286 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.696150    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c654d988-5jnqt" event={"ID":"0e881171-a183-4788-89d8-4da43869fc3c","Type":"ContainerDied","Data":"f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42"}
Oct 29 23:14:31.696286 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.696203    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c654d988-5jnqt" event={"ID":"0e881171-a183-4788-89d8-4da43869fc3c","Type":"ContainerDied","Data":"3dad951b6217976c1af259c4b0a21277c8c4809d41bc71ba12a13b8baa30d021"}
Oct 29 23:14:31.696286 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.696226    6040 scope.go:122] "RemoveContainer" containerID="f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42"
Oct 29 23:14:31.704179 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.704162    6040 scope.go:122] "RemoveContainer" containerID="f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42"
Oct 29 23:14:31.704486 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:14:31.704465    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42\": container with ID starting with f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42 not found: ID does not exist" containerID="f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42"
Oct 29 23:14:31.704535 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.704497    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42"} err="failed to get container status \"f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42\": rpc error: code = NotFound desc = could not find container \"f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42\": container with ID starting with f828a1ecdfc3337363fe4a91386aafa601db126f33b3b9c7b0f07187b83abc42 not found: ID does not exist"
Oct 29 23:14:31.746017 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.745985    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c654d988-5jnqt"]
Oct 29 23:14:31.758305 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:31.758276    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c654d988-5jnqt"]
Oct 29 23:14:32.421804 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:14:32.421775    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e881171-a183-4788-89d8-4da43869fc3c" path="/var/lib/kubelet/pods/0e881171-a183-4788-89d8-4da43869fc3c/volumes"
Oct 29 23:14:59.778065 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:14:59.778028    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:15:29.777254 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:15:29.777180    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:15:59.779217 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:15:59.779175    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:16:29.779156 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:16:29.779118    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:16:59.778228 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:16:59.778148    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:17:29.778013 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:17:29.777968    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:17:59.778582 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:17:59.778543    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:18:29.777818 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:18:29.777772    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:18:57.362188 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.362152    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"]
Oct 29 23:18:57.362837 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.362541    6040 cpu_manager.go:414] "containerMap: removing container" podUID="8ecd186b-7077-4136-82cf-a9804d85052c" containerName="console"
Oct 29 23:18:57.362837 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.362559    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ecd186b-7077-4136-82cf-a9804d85052c" containerName="console"
Oct 29 23:18:57.362837 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.362574    6040 cpu_manager.go:414] "containerMap: removing container" podUID="0e881171-a183-4788-89d8-4da43869fc3c" containerName="console"
Oct 29 23:18:57.362837 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.362581    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e881171-a183-4788-89d8-4da43869fc3c" containerName="console"
Oct 29 23:18:57.362837 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.362668    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ecd186b-7077-4136-82cf-a9804d85052c" containerName="console"
Oct 29 23:18:57.362837 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.362683    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e881171-a183-4788-89d8-4da43869fc3c" containerName="console"
Oct 29 23:18:57.365656 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.365635    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:18:57.392343 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.392311    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 29 23:18:57.392465 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.392439    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 29 23:18:57.392546 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.392529    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-8c9jv\""
Oct 29 23:18:57.401131 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.401105    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"]
Oct 29 23:18:57.503697 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.503667    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:18:57.503887 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.503720    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:18:57.503887 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.503762    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g7xr\" (UniqueName: \"kubernetes.io/projected/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-kube-api-access-7g7xr\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:18:57.605039 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.605007    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:18:57.605225 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.605060    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:18:57.605291 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.605225    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g7xr\" (UniqueName: \"kubernetes.io/projected/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-kube-api-access-7g7xr\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:18:57.605444 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.605418    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:18:57.605511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.605438    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:18:57.628810 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.628750    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g7xr\" (UniqueName: \"kubernetes.io/projected/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-kube-api-access-7g7xr\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:18:57.680367 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.680338    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:18:57.974096 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.974066    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"]
Oct 29 23:18:57.977912 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:18:57.977874    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe3cf66_6508_4615_8d10_0ebaec9a79f7.slice/crio-cb5d6ae4fed849ba77f32eaddd8362dd494d88e87750421b0da16314eb246d6e WatchSource:0}: Error finding container cb5d6ae4fed849ba77f32eaddd8362dd494d88e87750421b0da16314eb246d6e: Status 404 returned error can't find the container with id cb5d6ae4fed849ba77f32eaddd8362dd494d88e87750421b0da16314eb246d6e
Oct 29 23:18:57.980048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:57.980017    6040 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 29 23:18:58.533503 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:18:58.533465    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq" event={"ID":"1fe3cf66-6508-4615-8d10-0ebaec9a79f7","Type":"ContainerStarted","Data":"cb5d6ae4fed849ba77f32eaddd8362dd494d88e87750421b0da16314eb246d6e"}
Oct 29 23:18:59.782876 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:18:59.782830    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:19:03.553450 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:03.553416    6040 generic.go:356] "Generic (PLEG): container finished" podID="1fe3cf66-6508-4615-8d10-0ebaec9a79f7" containerID="2fce31d396632d57766c60b8665b1fa35240a4112e1987376a220428107ed921" exitCode=0
Oct 29 23:19:03.553884 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:03.553458    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq" event={"ID":"1fe3cf66-6508-4615-8d10-0ebaec9a79f7","Type":"ContainerDied","Data":"2fce31d396632d57766c60b8665b1fa35240a4112e1987376a220428107ed921"}
Oct 29 23:19:06.566502 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:06.566470    6040 generic.go:356] "Generic (PLEG): container finished" podID="1fe3cf66-6508-4615-8d10-0ebaec9a79f7" containerID="e52f124a3d61225ba48ab4bddc379ecee369ff0efb638310b97c87dad4d14281" exitCode=0
Oct 29 23:19:06.566906 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:06.566560    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq" event={"ID":"1fe3cf66-6508-4615-8d10-0ebaec9a79f7","Type":"ContainerDied","Data":"e52f124a3d61225ba48ab4bddc379ecee369ff0efb638310b97c87dad4d14281"}
Oct 29 23:19:12.591478 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:12.591443    6040 generic.go:356] "Generic (PLEG): container finished" podID="1fe3cf66-6508-4615-8d10-0ebaec9a79f7" containerID="cd8b345985e4c29e2522c0bdb1c0161335db7e3baa70f45297cfa784fd6eaf6b" exitCode=0
Oct 29 23:19:12.591883 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:12.591518    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq" event={"ID":"1fe3cf66-6508-4615-8d10-0ebaec9a79f7","Type":"ContainerDied","Data":"cd8b345985e4c29e2522c0bdb1c0161335db7e3baa70f45297cfa784fd6eaf6b"}
Oct 29 23:19:13.717032 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:13.717005    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:19:13.850273 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:13.850183    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-kube-api-access-7g7xr\" (UniqueName: \"kubernetes.io/projected/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-kube-api-access-7g7xr\") pod \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") "
Oct 29 23:19:13.850273 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:13.850231    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-util\" (UniqueName: \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-util\") pod \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") "
Oct 29 23:19:13.850501 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:13.850300    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-bundle\") pod \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\" (UID: \"1fe3cf66-6508-4615-8d10-0ebaec9a79f7\") "
Oct 29 23:19:13.850944 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:13.850912    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-bundle" pod "1fe3cf66-6508-4615-8d10-0ebaec9a79f7" (UID: "1fe3cf66-6508-4615-8d10-0ebaec9a79f7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:19:13.852472 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:13.852449    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-kube-api-access-7g7xr" pod "1fe3cf66-6508-4615-8d10-0ebaec9a79f7" (UID: "1fe3cf66-6508-4615-8d10-0ebaec9a79f7"). InnerVolumeSpecName "kube-api-access-7g7xr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:19:13.854055 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:13.854035    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-util" pod "1fe3cf66-6508-4615-8d10-0ebaec9a79f7" (UID: "1fe3cf66-6508-4615-8d10-0ebaec9a79f7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:19:13.951160 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:13.951115    6040 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-bundle\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:19:13.951160 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:13.951152    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g7xr\" (UniqueName: \"kubernetes.io/projected/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-kube-api-access-7g7xr\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:19:13.951160 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:13.951164    6040 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe3cf66-6508-4615-8d10-0ebaec9a79f7-util\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:19:14.600097 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:14.600070    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq"
Oct 29 23:19:14.600256 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:14.600069    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c46tfq" event={"ID":"1fe3cf66-6508-4615-8d10-0ebaec9a79f7","Type":"ContainerDied","Data":"cb5d6ae4fed849ba77f32eaddd8362dd494d88e87750421b0da16314eb246d6e"}
Oct 29 23:19:14.600256 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:14.600186    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb5d6ae4fed849ba77f32eaddd8362dd494d88e87750421b0da16314eb246d6e"
Oct 29 23:19:20.600542 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.600512    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb"]
Oct 29 23:19:20.601043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.600947    6040 cpu_manager.go:414] "containerMap: removing container" podUID="1fe3cf66-6508-4615-8d10-0ebaec9a79f7" containerName="extract"
Oct 29 23:19:20.601043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.600964    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fe3cf66-6508-4615-8d10-0ebaec9a79f7" containerName="extract"
Oct 29 23:19:20.601043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.600986    6040 cpu_manager.go:414] "containerMap: removing container" podUID="1fe3cf66-6508-4615-8d10-0ebaec9a79f7" containerName="util"
Oct 29 23:19:20.601043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.600993    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fe3cf66-6508-4615-8d10-0ebaec9a79f7" containerName="util"
Oct 29 23:19:20.601043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.601032    6040 cpu_manager.go:414] "containerMap: removing container" podUID="1fe3cf66-6508-4615-8d10-0ebaec9a79f7" containerName="pull"
Oct 29 23:19:20.601043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.601039    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fe3cf66-6508-4615-8d10-0ebaec9a79f7" containerName="pull"
Oct 29 23:19:20.601333 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.601096    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fe3cf66-6508-4615-8d10-0ebaec9a79f7" containerName="extract"
Oct 29 23:19:20.604270 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.604246    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb"
Oct 29 23:19:20.615279 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.615257    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb"]
Oct 29 23:19:20.626622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.626591    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:19:20.626750 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.626605    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 29 23:19:20.626750 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.626597    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-tr4zr\""
Oct 29 23:19:20.692876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.692840    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5l8d\" (UniqueName: \"kubernetes.io/projected/df3d7119-e629-436a-9683-7be527f573b3-kube-api-access-t5l8d\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-vd2bb\" (UID: \"df3d7119-e629-436a-9683-7be527f573b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb"
Oct 29 23:19:20.693021 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.692895    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df3d7119-e629-436a-9683-7be527f573b3-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-vd2bb\" (UID: \"df3d7119-e629-436a-9683-7be527f573b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb"
Oct 29 23:19:20.793539 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.793506    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5l8d\" (UniqueName: \"kubernetes.io/projected/df3d7119-e629-436a-9683-7be527f573b3-kube-api-access-t5l8d\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-vd2bb\" (UID: \"df3d7119-e629-436a-9683-7be527f573b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb"
Oct 29 23:19:20.793634 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.793579    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df3d7119-e629-436a-9683-7be527f573b3-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-vd2bb\" (UID: \"df3d7119-e629-436a-9683-7be527f573b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb"
Oct 29 23:19:20.794019 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.794000    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df3d7119-e629-436a-9683-7be527f573b3-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-vd2bb\" (UID: \"df3d7119-e629-436a-9683-7be527f573b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb"
Oct 29 23:19:20.817242 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.817213    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5l8d\" (UniqueName: \"kubernetes.io/projected/df3d7119-e629-436a-9683-7be527f573b3-kube-api-access-t5l8d\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-vd2bb\" (UID: \"df3d7119-e629-436a-9683-7be527f573b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb"
Oct 29 23:19:20.916029 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:20.916002    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb"
Oct 29 23:19:21.237855 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:21.237782    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb"]
Oct 29 23:19:21.241576 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:19:21.241548    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3d7119_e629_436a_9683_7be527f573b3.slice/crio-ed8e54b9b27a7498e6edd590e7485715f624484880edf4f819941f9899ddd14d WatchSource:0}: Error finding container ed8e54b9b27a7498e6edd590e7485715f624484880edf4f819941f9899ddd14d: Status 404 returned error can't find the container with id ed8e54b9b27a7498e6edd590e7485715f624484880edf4f819941f9899ddd14d
Oct 29 23:19:21.623591 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:21.623513    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb" event={"ID":"df3d7119-e629-436a-9683-7be527f573b3","Type":"ContainerStarted","Data":"ed8e54b9b27a7498e6edd590e7485715f624484880edf4f819941f9899ddd14d"}
Oct 29 23:19:23.631927 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:23.631889    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb" event={"ID":"df3d7119-e629-436a-9683-7be527f573b3","Type":"ContainerStarted","Data":"68837d1ab7e79faeb13505e4afd73f7424b0db9ae76b9f83262f16285374ab54"}
Oct 29 23:19:23.708633 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:23.708580    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-vd2bb" podStartSLOduration=2.270125535 podStartE2EDuration="3.708566035s" totalImagesPullingTime="1.4384405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:19:23.695958488 +0000 UTC m=+343.391067367" watchObservedRunningTime="2026-10-29 23:19:23.708566035 +0000 UTC m=+343.403674910"
Oct 29 23:19:24.665949 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.665918    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"]
Oct 29 23:19:24.669296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.669275    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"
Oct 29 23:19:24.684664 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.684646    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-c9cdn\""
Oct 29 23:19:24.696690 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.696670    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 29 23:19:24.696818 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.696762    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 29 23:19:24.697519 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.697497    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"]
Oct 29 23:19:24.718445 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.718420    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f27c9\" (UniqueName: \"kubernetes.io/projected/bf347111-37c9-481c-b4b1-a59fb1bf3422-kube-api-access-f27c9\") pod \"cert-manager-webhook-5dbbbf75cc-slh2h\" (UID: \"bf347111-37c9-481c-b4b1-a59fb1bf3422\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"
Oct 29 23:19:24.718542 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.718456    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf347111-37c9-481c-b4b1-a59fb1bf3422-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-slh2h\" (UID: \"bf347111-37c9-481c-b4b1-a59fb1bf3422\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"
Oct 29 23:19:24.819052 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.819019    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f27c9\" (UniqueName: \"kubernetes.io/projected/bf347111-37c9-481c-b4b1-a59fb1bf3422-kube-api-access-f27c9\") pod \"cert-manager-webhook-5dbbbf75cc-slh2h\" (UID: \"bf347111-37c9-481c-b4b1-a59fb1bf3422\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"
Oct 29 23:19:24.819183 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.819058    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf347111-37c9-481c-b4b1-a59fb1bf3422-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-slh2h\" (UID: \"bf347111-37c9-481c-b4b1-a59fb1bf3422\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"
Oct 29 23:19:24.847696 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.847663    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf347111-37c9-481c-b4b1-a59fb1bf3422-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-slh2h\" (UID: \"bf347111-37c9-481c-b4b1-a59fb1bf3422\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"
Oct 29 23:19:24.847891 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.847870    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f27c9\" (UniqueName: \"kubernetes.io/projected/bf347111-37c9-481c-b4b1-a59fb1bf3422-kube-api-access-f27c9\") pod \"cert-manager-webhook-5dbbbf75cc-slh2h\" (UID: \"bf347111-37c9-481c-b4b1-a59fb1bf3422\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"
Oct 29 23:19:24.996729 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:24.996652    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"
Oct 29 23:19:25.062646 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.062449    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"]
Oct 29 23:19:25.070289 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.070256    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:25.085192 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.085169    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"]
Oct 29 23:19:25.085452 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.085413    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"61b1ff68c7255ee5c5329fe19a5d96293a9578-ce91a85d-dockercfg-97j9t\""
Oct 29 23:19:25.085452 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.085419    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 29 23:19:25.085593 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.085464    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 29 23:19:25.121171 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.121139    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:25.121394 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.121172    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hqr5\" (UniqueName: \"kubernetes.io/projected/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-kube-api-access-8hqr5\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:25.121394 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.121199    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:25.221609 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.221577    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:25.221769 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.221615    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hqr5\" (UniqueName: \"kubernetes.io/projected/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-kube-api-access-8hqr5\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:25.221769 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.221640    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:25.222052 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.222031    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:25.222092 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.222065    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:25.244899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.244873    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hqr5\" (UniqueName: \"kubernetes.io/projected/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-kube-api-access-8hqr5\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:25.292170 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.292104    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"]
Oct 29 23:19:25.295905 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:19:25.295881    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf347111_37c9_481c_b4b1_a59fb1bf3422.slice/crio-87994a8c026ff0f6e3d244c797ede06c9514144efb1ea9b8d538e420dcfc08fc WatchSource:0}: Error finding container 87994a8c026ff0f6e3d244c797ede06c9514144efb1ea9b8d538e420dcfc08fc: Status 404 returned error can't find the container with id 87994a8c026ff0f6e3d244c797ede06c9514144efb1ea9b8d538e420dcfc08fc
Oct 29 23:19:25.385583 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.385549    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:25.641455 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.641425    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h" event={"ID":"bf347111-37c9-481c-b4b1-a59fb1bf3422","Type":"ContainerStarted","Data":"87994a8c026ff0f6e3d244c797ede06c9514144efb1ea9b8d538e420dcfc08fc"}
Oct 29 23:19:25.677721 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:25.677682    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"]
Oct 29 23:19:25.680780 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:19:25.680752    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbeeb93b_a747_4cda_b8b5_cca6e0de46ec.slice/crio-75edbc76e48418ecb2c9b785a1e213e9ec745ff0705d30703b3ba29b628a5d3d WatchSource:0}: Error finding container 75edbc76e48418ecb2c9b785a1e213e9ec745ff0705d30703b3ba29b628a5d3d: Status 404 returned error can't find the container with id 75edbc76e48418ecb2c9b785a1e213e9ec745ff0705d30703b3ba29b628a5d3d
Oct 29 23:19:26.647646 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:26.647610    6040 generic.go:356] "Generic (PLEG): container finished" podID="bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" containerID="510a2c62c241ab3b6b44040e9cab83ea085e81c09e9a24d9d63124fd131cffe4" exitCode=0
Oct 29 23:19:26.647845 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:26.647652    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw" event={"ID":"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec","Type":"ContainerDied","Data":"510a2c62c241ab3b6b44040e9cab83ea085e81c09e9a24d9d63124fd131cffe4"}
Oct 29 23:19:26.647845 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:26.647696    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw" event={"ID":"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec","Type":"ContainerStarted","Data":"75edbc76e48418ecb2c9b785a1e213e9ec745ff0705d30703b3ba29b628a5d3d"}
Oct 29 23:19:28.657056 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:28.657016    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h" event={"ID":"bf347111-37c9-481c-b4b1-a59fb1bf3422","Type":"ContainerStarted","Data":"35951c8d99485b60e576fc71efe9061c0568e5ebcdd94ae0764bd6b66bb6f77b"}
Oct 29 23:19:28.657520 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:28.657140    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"
Oct 29 23:19:28.702000 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:28.701948    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h" podStartSLOduration=2.07177599 podStartE2EDuration="4.701933772s" totalImagesPullingTime="2.630157782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:19:28.696688468 +0000 UTC m=+348.391797344" watchObservedRunningTime="2026-10-29 23:19:28.701933772 +0000 UTC m=+348.397042650"
Oct 29 23:19:29.663174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:29.663139    6040 generic.go:356] "Generic (PLEG): container finished" podID="bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" containerID="0920f0cf4e134364430fc91b2bcfbd123eda7b7857635b6d36d24c6e94eff88d" exitCode=0
Oct 29 23:19:29.663670 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:29.663188    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw" event={"ID":"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec","Type":"ContainerDied","Data":"0920f0cf4e134364430fc91b2bcfbd123eda7b7857635b6d36d24c6e94eff88d"}
Oct 29 23:19:29.778477 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:19:29.778445    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:19:30.673138 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:30.673104    6040 generic.go:356] "Generic (PLEG): container finished" podID="bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" containerID="3dba6075d5256d80dc5e5cf2d9e40db647bfda2d4e51c163256c1ad382381942" exitCode=0
Oct 29 23:19:30.673536 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:30.673168    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw" event={"ID":"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec","Type":"ContainerDied","Data":"3dba6075d5256d80dc5e5cf2d9e40db647bfda2d4e51c163256c1ad382381942"}
Oct 29 23:19:31.425077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.425043    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8"]
Oct 29 23:19:31.429594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.429577    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8"
Oct 29 23:19:31.446552 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.446523    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8"]
Oct 29 23:19:31.457455 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.457432    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qlsfl\""
Oct 29 23:19:31.464633 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.464609    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbb7j\" (UniqueName: \"kubernetes.io/projected/8b0fa476-c652-4693-a7e6-69a269969ebe-kube-api-access-rbb7j\") pod \"cert-manager-cainjector-5c57bbd4fc-7jhp8\" (UID: \"8b0fa476-c652-4693-a7e6-69a269969ebe\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8"
Oct 29 23:19:31.464772 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.464669    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b0fa476-c652-4693-a7e6-69a269969ebe-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-7jhp8\" (UID: \"8b0fa476-c652-4693-a7e6-69a269969ebe\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8"
Oct 29 23:19:31.565126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.565093    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbb7j\" (UniqueName: \"kubernetes.io/projected/8b0fa476-c652-4693-a7e6-69a269969ebe-kube-api-access-rbb7j\") pod \"cert-manager-cainjector-5c57bbd4fc-7jhp8\" (UID: \"8b0fa476-c652-4693-a7e6-69a269969ebe\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8"
Oct 29 23:19:31.565269 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.565174    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b0fa476-c652-4693-a7e6-69a269969ebe-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-7jhp8\" (UID: \"8b0fa476-c652-4693-a7e6-69a269969ebe\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8"
Oct 29 23:19:31.591182 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.591152    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b0fa476-c652-4693-a7e6-69a269969ebe-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-7jhp8\" (UID: \"8b0fa476-c652-4693-a7e6-69a269969ebe\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8"
Oct 29 23:19:31.591302 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.591217    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbb7j\" (UniqueName: \"kubernetes.io/projected/8b0fa476-c652-4693-a7e6-69a269969ebe-kube-api-access-rbb7j\") pod \"cert-manager-cainjector-5c57bbd4fc-7jhp8\" (UID: \"8b0fa476-c652-4693-a7e6-69a269969ebe\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8"
Oct 29 23:19:31.742148 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.742111    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8"
Oct 29 23:19:31.795559 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.795454    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:31.867166 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.867134    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-kube-api-access-8hqr5\" (UniqueName: \"kubernetes.io/projected/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-kube-api-access-8hqr5\") pod \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") "
Oct 29 23:19:31.869035 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.869013    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-kube-api-access-8hqr5" pod "bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" (UID: "bbeeb93b-a747-4cda-b8b5-cca6e0de46ec"). InnerVolumeSpecName "kube-api-access-8hqr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:19:31.967496 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.967428    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-bundle\") pod \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") "
Oct 29 23:19:31.967622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.967519    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-util\" (UniqueName: \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-util\") pod \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\" (UID: \"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec\") "
Oct 29 23:19:31.967688 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.967670    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hqr5\" (UniqueName: \"kubernetes.io/projected/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-kube-api-access-8hqr5\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:19:31.967833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.967810    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-bundle" pod "bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" (UID: "bbeeb93b-a747-4cda-b8b5-cca6e0de46ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:19:31.971907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:31.971888    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-util" pod "bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" (UID: "bbeeb93b-a747-4cda-b8b5-cca6e0de46ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:19:32.039572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:32.039526    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8"]
Oct 29 23:19:32.042759 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:19:32.042719    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b0fa476_c652_4693_a7e6_69a269969ebe.slice/crio-7724a4cc60580f9632905b31971dd71c342c5a23f7ef5249418b12e077ef5c9d WatchSource:0}: Error finding container 7724a4cc60580f9632905b31971dd71c342c5a23f7ef5249418b12e077ef5c9d: Status 404 returned error can't find the container with id 7724a4cc60580f9632905b31971dd71c342c5a23f7ef5249418b12e077ef5c9d
Oct 29 23:19:32.068598 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:32.068577    6040 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-util\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:19:32.068598 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:32.068598    6040 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbeeb93b-a747-4cda-b8b5-cca6e0de46ec-bundle\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:19:32.682340 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:32.682303    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8" event={"ID":"8b0fa476-c652-4693-a7e6-69a269969ebe","Type":"ContainerStarted","Data":"55f6b8d3275c088a7123b743bfaf754f3b1ec436312525a324cdc59324c4c047"}
Oct 29 23:19:32.682524 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:32.682355    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8" event={"ID":"8b0fa476-c652-4693-a7e6-69a269969ebe","Type":"ContainerStarted","Data":"7724a4cc60580f9632905b31971dd71c342c5a23f7ef5249418b12e077ef5c9d"}
Oct 29 23:19:32.684180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:32.684151    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw" event={"ID":"bbeeb93b-a747-4cda-b8b5-cca6e0de46ec","Type":"ContainerDied","Data":"75edbc76e48418ecb2c9b785a1e213e9ec745ff0705d30703b3ba29b628a5d3d"}
Oct 29 23:19:32.684180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:32.684172    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85xg2kw"
Oct 29 23:19:32.684346 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:32.684188    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75edbc76e48418ecb2c9b785a1e213e9ec745ff0705d30703b3ba29b628a5d3d"
Oct 29 23:19:32.721893 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:32.721850    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jhp8" podStartSLOduration=1.7218373470000001 podStartE2EDuration="1.721837347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:19:32.721255618 +0000 UTC m=+352.416364499" watchObservedRunningTime="2026-10-29 23:19:32.721837347 +0000 UTC m=+352.416946216"
Oct 29 23:19:34.665271 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:34.665239    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h"
Oct 29 23:19:38.930388 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:38.930353    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-wglf2"]
Oct 29 23:19:38.930876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:38.930755    6040 cpu_manager.go:414] "containerMap: removing container" podUID="bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" containerName="pull"
Oct 29 23:19:38.930876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:38.930772    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" containerName="pull"
Oct 29 23:19:38.930876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:38.930806    6040 cpu_manager.go:414] "containerMap: removing container" podUID="bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" containerName="extract"
Oct 29 23:19:38.930876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:38.930814    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" containerName="extract"
Oct 29 23:19:38.930876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:38.930830    6040 cpu_manager.go:414] "containerMap: removing container" podUID="bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" containerName="util"
Oct 29 23:19:38.930876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:38.930836    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" containerName="util"
Oct 29 23:19:38.931184 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:38.930901    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbeeb93b-a747-4cda-b8b5-cca6e0de46ec" containerName="extract"
Oct 29 23:19:38.973946 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:38.973912    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-wglf2"]
Oct 29 23:19:38.974087 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:38.974028    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-wglf2"
Oct 29 23:19:39.006487 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.006464    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:19:39.006487 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.006482    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Oct 29 23:19:39.006763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.006726    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-m28ls\""
Oct 29 23:19:39.014179 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.014160    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd87fa60-9886-4d17-8a72-c2e12cc04d34-tmp\") pod \"openshift-lws-operator-8465994787-wglf2\" (UID: \"fd87fa60-9886-4d17-8a72-c2e12cc04d34\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wglf2"
Oct 29 23:19:39.014256 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.014206    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nll26\" (UniqueName: \"kubernetes.io/projected/fd87fa60-9886-4d17-8a72-c2e12cc04d34-kube-api-access-nll26\") pod \"openshift-lws-operator-8465994787-wglf2\" (UID: \"fd87fa60-9886-4d17-8a72-c2e12cc04d34\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wglf2"
Oct 29 23:19:39.115290 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.115253    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd87fa60-9886-4d17-8a72-c2e12cc04d34-tmp\") pod \"openshift-lws-operator-8465994787-wglf2\" (UID: \"fd87fa60-9886-4d17-8a72-c2e12cc04d34\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wglf2"
Oct 29 23:19:39.115480 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.115332    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nll26\" (UniqueName: \"kubernetes.io/projected/fd87fa60-9886-4d17-8a72-c2e12cc04d34-kube-api-access-nll26\") pod \"openshift-lws-operator-8465994787-wglf2\" (UID: \"fd87fa60-9886-4d17-8a72-c2e12cc04d34\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wglf2"
Oct 29 23:19:39.115641 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.115618    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd87fa60-9886-4d17-8a72-c2e12cc04d34-tmp\") pod \"openshift-lws-operator-8465994787-wglf2\" (UID: \"fd87fa60-9886-4d17-8a72-c2e12cc04d34\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wglf2"
Oct 29 23:19:39.138723 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.138691    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nll26\" (UniqueName: \"kubernetes.io/projected/fd87fa60-9886-4d17-8a72-c2e12cc04d34-kube-api-access-nll26\") pod \"openshift-lws-operator-8465994787-wglf2\" (UID: \"fd87fa60-9886-4d17-8a72-c2e12cc04d34\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-wglf2"
Oct 29 23:19:39.296322 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.296245    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-wglf2"
Oct 29 23:19:39.608798 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.608713    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-wglf2"]
Oct 29 23:19:39.611989 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:19:39.611964    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd87fa60_9886_4d17_8a72_c2e12cc04d34.slice/crio-cf0cb8fd38d3c8a28e6fc67712702181a48bf23c017589f72db13f09ddb0afd6 WatchSource:0}: Error finding container cf0cb8fd38d3c8a28e6fc67712702181a48bf23c017589f72db13f09ddb0afd6: Status 404 returned error can't find the container with id cf0cb8fd38d3c8a28e6fc67712702181a48bf23c017589f72db13f09ddb0afd6
Oct 29 23:19:39.710911 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:39.710873    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-wglf2" event={"ID":"fd87fa60-9886-4d17-8a72-c2e12cc04d34","Type":"ContainerStarted","Data":"cf0cb8fd38d3c8a28e6fc67712702181a48bf23c017589f72db13f09ddb0afd6"}
Oct 29 23:19:42.725131 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:42.725099    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-wglf2" event={"ID":"fd87fa60-9886-4d17-8a72-c2e12cc04d34","Type":"ContainerStarted","Data":"1306f9c7f67f15810b5b724b35577f1f52f3ef0d38efa950e064461cb66e6648"}
Oct 29 23:19:42.765977 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:42.765927    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-8465994787-wglf2" podStartSLOduration=2.5235912579999997 podStartE2EDuration="4.765915411s" totalImagesPullingTime="2.242324153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:19:42.765555469 +0000 UTC m=+362.460664348" watchObservedRunningTime="2026-10-29 23:19:42.765915411 +0000 UTC m=+362.461024290"
Oct 29 23:19:45.480365 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.480328    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"]
Oct 29 23:19:45.483399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.483382    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:45.497959 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.497935    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 29 23:19:45.509824 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.509801    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 29 23:19:45.509947 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.509839    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-d7dw7\""
Oct 29 23:19:45.510782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.510763    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"]
Oct 29 23:19:45.559508 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.559482    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:45.559614 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.559518    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:45.559614 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.559550    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk5x2\" (UniqueName: \"kubernetes.io/projected/425caaf4-7064-4beb-9900-c18d403ab5ac-kube-api-access-nk5x2\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:45.660204 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.660173    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:45.660337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.660223    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk5x2\" (UniqueName: \"kubernetes.io/projected/425caaf4-7064-4beb-9900-c18d403ab5ac-kube-api-access-nk5x2\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:45.660337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.660305    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:45.660631 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.660612    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:45.660671 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.660627    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:45.683960 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.683931    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk5x2\" (UniqueName: \"kubernetes.io/projected/425caaf4-7064-4beb-9900-c18d403ab5ac-kube-api-access-nk5x2\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:45.797734 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:45.797653    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:46.092149 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:46.092073    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"]
Oct 29 23:19:46.095063 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:19:46.095035    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod425caaf4_7064_4beb_9900_c18d403ab5ac.slice/crio-17454efb0190925ab83f2189299231765cdf3bc356251f637b45a2a8007c94ca WatchSource:0}: Error finding container 17454efb0190925ab83f2189299231765cdf3bc356251f637b45a2a8007c94ca: Status 404 returned error can't find the container with id 17454efb0190925ab83f2189299231765cdf3bc356251f637b45a2a8007c94ca
Oct 29 23:19:46.742438 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:46.742407    6040 generic.go:356] "Generic (PLEG): container finished" podID="425caaf4-7064-4beb-9900-c18d403ab5ac" containerID="f535053c5ecb500f00b841455617fc856f242b7ce89ae80cdfe1795343fd0076" exitCode=0
Oct 29 23:19:46.742866 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:46.742467    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v" event={"ID":"425caaf4-7064-4beb-9900-c18d403ab5ac","Type":"ContainerDied","Data":"f535053c5ecb500f00b841455617fc856f242b7ce89ae80cdfe1795343fd0076"}
Oct 29 23:19:46.742866 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:46.742501    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v" event={"ID":"425caaf4-7064-4beb-9900-c18d403ab5ac","Type":"ContainerStarted","Data":"17454efb0190925ab83f2189299231765cdf3bc356251f637b45a2a8007c94ca"}
Oct 29 23:19:47.747941 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:47.747905    6040 generic.go:356] "Generic (PLEG): container finished" podID="425caaf4-7064-4beb-9900-c18d403ab5ac" containerID="3383519d8f154c0d15e17b7ea8a4eb9e994fcacfa4039b314b1b1ff15f4a3e35" exitCode=0
Oct 29 23:19:47.748394 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:47.747987    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v" event={"ID":"425caaf4-7064-4beb-9900-c18d403ab5ac","Type":"ContainerDied","Data":"3383519d8f154c0d15e17b7ea8a4eb9e994fcacfa4039b314b1b1ff15f4a3e35"}
Oct 29 23:19:48.753822 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:48.753786    6040 generic.go:356] "Generic (PLEG): container finished" podID="425caaf4-7064-4beb-9900-c18d403ab5ac" containerID="5f171d8cb9a77e15859a08ec692b51cd33c0c4538368e913806726400a3c7faa" exitCode=0
Oct 29 23:19:48.754207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:48.753865    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v" event={"ID":"425caaf4-7064-4beb-9900-c18d403ab5ac","Type":"ContainerDied","Data":"5f171d8cb9a77e15859a08ec692b51cd33c0c4538368e913806726400a3c7faa"}
Oct 29 23:19:49.876888 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:49.876859    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:49.888867 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:49.888847    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-util\" (UniqueName: \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-util\") pod \"425caaf4-7064-4beb-9900-c18d403ab5ac\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") "
Oct 29 23:19:49.888937 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:49.888888    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/425caaf4-7064-4beb-9900-c18d403ab5ac-kube-api-access-nk5x2\" (UniqueName: \"kubernetes.io/projected/425caaf4-7064-4beb-9900-c18d403ab5ac-kube-api-access-nk5x2\") pod \"425caaf4-7064-4beb-9900-c18d403ab5ac\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") "
Oct 29 23:19:49.888937 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:49.888911    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-bundle\" (UniqueName: \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-bundle\") pod \"425caaf4-7064-4beb-9900-c18d403ab5ac\" (UID: \"425caaf4-7064-4beb-9900-c18d403ab5ac\") "
Oct 29 23:19:49.889475 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:49.889445    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-bundle" pod "425caaf4-7064-4beb-9900-c18d403ab5ac" (UID: "425caaf4-7064-4beb-9900-c18d403ab5ac"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:19:49.890878 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:49.890855    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/425caaf4-7064-4beb-9900-c18d403ab5ac-kube-api-access-nk5x2" pod "425caaf4-7064-4beb-9900-c18d403ab5ac" (UID: "425caaf4-7064-4beb-9900-c18d403ab5ac"). InnerVolumeSpecName "kube-api-access-nk5x2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:19:49.894005 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:49.893980    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-util" pod "425caaf4-7064-4beb-9900-c18d403ab5ac" (UID: "425caaf4-7064-4beb-9900-c18d403ab5ac"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:19:49.989934 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:49.989892    6040 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-bundle\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:19:49.989934 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:49.989926    6040 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/425caaf4-7064-4beb-9900-c18d403ab5ac-util\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:19:49.989934 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:49.989935    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk5x2\" (UniqueName: \"kubernetes.io/projected/425caaf4-7064-4beb-9900-c18d403ab5ac-kube-api-access-nk5x2\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:19:50.763805 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:50.763710    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v" event={"ID":"425caaf4-7064-4beb-9900-c18d403ab5ac","Type":"ContainerDied","Data":"17454efb0190925ab83f2189299231765cdf3bc356251f637b45a2a8007c94ca"}
Oct 29 23:19:50.763805 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:50.763778    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17454efb0190925ab83f2189299231765cdf3bc356251f637b45a2a8007c94ca"
Oct 29 23:19:50.763805 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:50.763778    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71mhv2v"
Oct 29 23:19:51.948367 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:51.948335    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-848684cc69-h966r"]
Oct 29 23:19:51.948761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:51.948612    6040 cpu_manager.go:414] "containerMap: removing container" podUID="425caaf4-7064-4beb-9900-c18d403ab5ac" containerName="extract"
Oct 29 23:19:51.948761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:51.948624    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="425caaf4-7064-4beb-9900-c18d403ab5ac" containerName="extract"
Oct 29 23:19:51.948761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:51.948638    6040 cpu_manager.go:414] "containerMap: removing container" podUID="425caaf4-7064-4beb-9900-c18d403ab5ac" containerName="util"
Oct 29 23:19:51.948761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:51.948643    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="425caaf4-7064-4beb-9900-c18d403ab5ac" containerName="util"
Oct 29 23:19:51.948761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:51.948665    6040 cpu_manager.go:414] "containerMap: removing container" podUID="425caaf4-7064-4beb-9900-c18d403ab5ac" containerName="pull"
Oct 29 23:19:51.948761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:51.948671    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="425caaf4-7064-4beb-9900-c18d403ab5ac" containerName="pull"
Oct 29 23:19:51.948761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:51.948723    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="425caaf4-7064-4beb-9900-c18d403ab5ac" containerName="extract"
Oct 29 23:19:51.951650 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:51.951632    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:51.971519 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:51.971494    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-848684cc69-h966r"]
Oct 29 23:19:52.002059 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.002024    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx5sk\" (UniqueName: \"kubernetes.io/projected/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-kube-api-access-hx5sk\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.002212 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.002077    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-cert\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.002212 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.002099    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-manager-config\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.002212 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.002170    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-metrics-cert\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.049161 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.049130    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Oct 29 23:19:52.049315 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.049206    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9j7fg\""
Oct 29 23:19:52.049315 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.049212    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Oct 29 23:19:52.049315 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.049235    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Oct 29 23:19:52.102717 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.102685    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-cert\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.102884 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.102724    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-manager-config\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.102884 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.102776    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-metrics-cert\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.102884 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.102808    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx5sk\" (UniqueName: \"kubernetes.io/projected/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-kube-api-access-hx5sk\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.103450 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.103426    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-manager-config\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.105357 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.105336    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-cert\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.106364 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.106343    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-metrics-cert\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.126024 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.125994    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx5sk\" (UniqueName: \"kubernetes.io/projected/4fdc022d-11d2-4f8a-81af-4d1c39f0a280-kube-api-access-hx5sk\") pod \"lws-controller-manager-848684cc69-h966r\" (UID: \"4fdc022d-11d2-4f8a-81af-4d1c39f0a280\") " pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.263326 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.263243    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:52.558371 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.558303    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-848684cc69-h966r"]
Oct 29 23:19:52.561880 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:19:52.561854    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fdc022d_11d2_4f8a_81af_4d1c39f0a280.slice/crio-c6703147296ec4858c6ce1c06bbaac8711513fad2f9619aff57af544e8354c5d WatchSource:0}: Error finding container c6703147296ec4858c6ce1c06bbaac8711513fad2f9619aff57af544e8354c5d: Status 404 returned error can't find the container with id c6703147296ec4858c6ce1c06bbaac8711513fad2f9619aff57af544e8354c5d
Oct 29 23:19:52.777143 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:52.777111    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r" event={"ID":"4fdc022d-11d2-4f8a-81af-4d1c39f0a280","Type":"ContainerStarted","Data":"c6703147296ec4858c6ce1c06bbaac8711513fad2f9619aff57af544e8354c5d"}
Oct 29 23:19:55.791320 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:55.791285    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r" event={"ID":"4fdc022d-11d2-4f8a-81af-4d1c39f0a280","Type":"ContainerStarted","Data":"fdb05e5a07edc9247ead7d2ee014349e4b7a86ea27bed8cdd1ee77ed1e613a36"}
Oct 29 23:19:55.791714 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:55.791348    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:19:55.831752 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:19:55.831681    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r" podStartSLOduration=2.499794276 podStartE2EDuration="4.831663998s" totalImagesPullingTime="2.331869722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:19:55.831379616 +0000 UTC m=+375.526488508" watchObservedRunningTime="2026-10-29 23:19:55.831663998 +0000 UTC m=+375.526772881"
Oct 29 23:19:59.778591 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:19:59.778552    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:20:01.082848 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.082814    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"]
Oct 29 23:20:01.086346 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.086327    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.129309 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.129272    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"]
Oct 29 23:20:01.151293 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.151263    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 29 23:20:01.151553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.151536    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\""
Oct 29 23:20:01.151639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.151556    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-295kq\""
Oct 29 23:20:01.151687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.151663    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 29 23:20:01.158671 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.158658    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 29 23:20:01.158728 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.158710    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 29 23:20:01.163277 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.163261    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/175dfb73-992b-404b-a72d-562b052cba0a-tmp\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.163327 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.163289    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/175dfb73-992b-404b-a72d-562b052cba0a-manifests\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.163327 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.163306    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/175dfb73-992b-404b-a72d-562b052cba0a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.163387 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.163329    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/175dfb73-992b-404b-a72d-562b052cba0a-webhook-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.163438 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.163417    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/175dfb73-992b-404b-a72d-562b052cba0a-metrics-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.163478 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.163464    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdpd4\" (UniqueName: \"kubernetes.io/projected/175dfb73-992b-404b-a72d-562b052cba0a-kube-api-access-kdpd4\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.263831 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.263791    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/175dfb73-992b-404b-a72d-562b052cba0a-tmp\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.264010 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.263840    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/175dfb73-992b-404b-a72d-562b052cba0a-manifests\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.264010 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.263957    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/175dfb73-992b-404b-a72d-562b052cba0a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.264010 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.263993    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/175dfb73-992b-404b-a72d-562b052cba0a-webhook-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.264172 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.264043    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/175dfb73-992b-404b-a72d-562b052cba0a-metrics-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.264172 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.264079    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdpd4\" (UniqueName: \"kubernetes.io/projected/175dfb73-992b-404b-a72d-562b052cba0a-kube-api-access-kdpd4\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.264297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.264271    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/175dfb73-992b-404b-a72d-562b052cba0a-manifests\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.266042 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.266012    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/175dfb73-992b-404b-a72d-562b052cba0a-tmp\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.266304 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.266286    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/175dfb73-992b-404b-a72d-562b052cba0a-webhook-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.266383 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.266355    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/175dfb73-992b-404b-a72d-562b052cba0a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.266516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.266492    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/175dfb73-992b-404b-a72d-562b052cba0a-metrics-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.286876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.286853    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdpd4\" (UniqueName: \"kubernetes.io/projected/175dfb73-992b-404b-a72d-562b052cba0a-kube-api-access-kdpd4\") pod \"opendatahub-operator-controller-manager-57666cc899-t5nwr\" (UID: \"175dfb73-992b-404b-a72d-562b052cba0a\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.400988 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.400957    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:01.755309 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.755217    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"]
Oct 29 23:20:01.758305 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:20:01.758279    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod175dfb73_992b_404b_a72d_562b052cba0a.slice/crio-faccaad47ef1953152ea4d973e282f7ce8dacf4f4b07fc4a412d6f6091df2513 WatchSource:0}: Error finding container faccaad47ef1953152ea4d973e282f7ce8dacf4f4b07fc4a412d6f6091df2513: Status 404 returned error can't find the container with id faccaad47ef1953152ea4d973e282f7ce8dacf4f4b07fc4a412d6f6091df2513
Oct 29 23:20:01.815820 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:01.815789    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr" event={"ID":"175dfb73-992b-404b-a72d-562b052cba0a","Type":"ContainerStarted","Data":"faccaad47ef1953152ea4d973e282f7ce8dacf4f4b07fc4a412d6f6091df2513"}
Oct 29 23:20:04.831806 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:04.831695    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr" event={"ID":"175dfb73-992b-404b-a72d-562b052cba0a","Type":"ContainerStarted","Data":"1af834fdf2e38848f96c58915b7ef07f66c9fb4c1eb66a6fcc3580d55529fc60"}
Oct 29 23:20:05.838290 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:05.838258    6040 generic.go:356] "Generic (PLEG): container finished" podID="175dfb73-992b-404b-a72d-562b052cba0a" containerID="1af834fdf2e38848f96c58915b7ef07f66c9fb4c1eb66a6fcc3580d55529fc60" exitCode=0
Oct 29 23:20:05.838847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:05.838327    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr" event={"ID":"175dfb73-992b-404b-a72d-562b052cba0a","Type":"ContainerDied","Data":"1af834fdf2e38848f96c58915b7ef07f66c9fb4c1eb66a6fcc3580d55529fc60"}
Oct 29 23:20:06.798054 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:06.798017    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-848684cc69-h966r"
Oct 29 23:20:06.845611 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:06.845580    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr" event={"ID":"175dfb73-992b-404b-a72d-562b052cba0a","Type":"ContainerStarted","Data":"3ae2c9757abbad1198fe5ab74e7c2a099bf3f84c032a8f618a48299db15a3804"}
Oct 29 23:20:06.846031 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:06.845698    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:06.934667 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:06.934611    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr" podStartSLOduration=3.417687275 podStartE2EDuration="6.934597364s" totalImagesPullingTime="3.101884946s" totalInitContainerRuntime="415.025143ms" isStatefulPod=false podCreationTimestamp="2026-10-29 23:20:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:20:06.922656502 +0000 UTC m=+386.617765381" watchObservedRunningTime="2026-10-29 23:20:06.934597364 +0000 UTC m=+386.629706242"
Oct 29 23:20:17.852072 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:17.852038    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-t5nwr"
Oct 29 23:20:29.778436 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:20:29.778392    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:20:36.563848 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.563816    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"]
Oct 29 23:20:36.567917 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.567896    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.621001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.620970    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 29 23:20:36.621153 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.621062    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-6f677\""
Oct 29 23:20:36.622305 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.622285    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"]
Oct 29 23:20:36.667539 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.667509    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d3a9d783-e528-4069-aeab-358b1af624cb-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.667635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.667543    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3a9d783-e528-4069-aeab-358b1af624cb-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.667635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.667620    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmvms\" (UniqueName: \"kubernetes.io/projected/d3a9d783-e528-4069-aeab-358b1af624cb-kube-api-access-wmvms\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.667753 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.667723    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3a9d783-e528-4069-aeab-358b1af624cb-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.768558 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.768526    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmvms\" (UniqueName: \"kubernetes.io/projected/d3a9d783-e528-4069-aeab-358b1af624cb-kube-api-access-wmvms\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.768699 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.768565    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3a9d783-e528-4069-aeab-358b1af624cb-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.768761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.768693    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d3a9d783-e528-4069-aeab-358b1af624cb-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.768761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.768754    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3a9d783-e528-4069-aeab-358b1af624cb-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.769037 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.769018    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3a9d783-e528-4069-aeab-358b1af624cb-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.769073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.769024    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d3a9d783-e528-4069-aeab-358b1af624cb-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.769475 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.769458    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3a9d783-e528-4069-aeab-358b1af624cb-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.791653 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.791631    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmvms\" (UniqueName: \"kubernetes.io/projected/d3a9d783-e528-4069-aeab-358b1af624cb-kube-api-access-wmvms\") pod \"kserve-module-controller-manager-5cfb77bf8c-cdpft\" (UID: \"d3a9d783-e528-4069-aeab-358b1af624cb\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:36.880344 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:36.880273    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:37.188395 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:37.188206    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"]
Oct 29 23:20:37.191098 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:20:37.191067    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a9d783_e528_4069_aeab_358b1af624cb.slice/crio-2d1db9fe535675a2752f6e122276a8e8f2c893ce55a7c13cb9ccf0042227e0b5 WatchSource:0}: Error finding container 2d1db9fe535675a2752f6e122276a8e8f2c893ce55a7c13cb9ccf0042227e0b5: Status 404 returned error can't find the container with id 2d1db9fe535675a2752f6e122276a8e8f2c893ce55a7c13cb9ccf0042227e0b5
Oct 29 23:20:37.974607 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:37.974562    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft" event={"ID":"d3a9d783-e528-4069-aeab-358b1af624cb","Type":"ContainerStarted","Data":"2d1db9fe535675a2752f6e122276a8e8f2c893ce55a7c13cb9ccf0042227e0b5"}
Oct 29 23:20:39.064606 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.064568    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"]
Oct 29 23:20:39.069605 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.068666    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.084001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.083723    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 29 23:20:39.084001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.083840    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zdfj8\""
Oct 29 23:20:39.095235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.095213    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 29 23:20:39.095369 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.095242    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 29 23:20:39.096855 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.096836    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"]
Oct 29 23:20:39.190753 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.190700    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.190940 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.190766    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.190940 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.190847    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.190940 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.190875    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-token\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.190940 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.190901    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdqjs\" (UniqueName: \"kubernetes.io/projected/ed971fe7-998b-462e-bd88-3bb7001aa163-kube-api-access-xdqjs\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.191140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.190985    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.191140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.191055    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.191221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.191132    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-data\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.191221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.191169    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed971fe7-998b-462e-bd88-3bb7001aa163-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.292607 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.292572    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.292851 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.292627    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.292851 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.292657    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.292851 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.292682    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-token\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.292851 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.292710    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqjs\" (UniqueName: \"kubernetes.io/projected/ed971fe7-998b-462e-bd88-3bb7001aa163-kube-api-access-xdqjs\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.292851 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.292759    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.293138 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.292937    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.293138 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.292987    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.293138 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.293011    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-data\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.293138 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.293056    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed971fe7-998b-462e-bd88-3bb7001aa163-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.293138 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.293077    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.293354 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.293247    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.293354 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.293299    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-data\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.293837 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.293813    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed971fe7-998b-462e-bd88-3bb7001aa163-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.295208 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.295190    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.295208 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.295209    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.316171 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.316104    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdqjs\" (UniqueName: \"kubernetes.io/projected/ed971fe7-998b-462e-bd88-3bb7001aa163-kube-api-access-xdqjs\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.316284 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.316197    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed971fe7-998b-462e-bd88-3bb7001aa163-istio-token\") pod \"data-science-gateway-data-science-gateway-class-557c4f746fjz9lk\" (UID: \"ed971fe7-998b-462e-bd88-3bb7001aa163\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.383897 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.383860    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:39.700087 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.700056    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"]
Oct 29 23:20:39.703173 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:20:39.703146    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded971fe7_998b_462e_bd88_3bb7001aa163.slice/crio-d80ac685cee45397d58e331f1bf1d92f13339c602296f29ac66868e3d34b3051 WatchSource:0}: Error finding container d80ac685cee45397d58e331f1bf1d92f13339c602296f29ac66868e3d34b3051: Status 404 returned error can't find the container with id d80ac685cee45397d58e331f1bf1d92f13339c602296f29ac66868e3d34b3051
Oct 29 23:20:39.984989 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.984880    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft" event={"ID":"d3a9d783-e528-4069-aeab-358b1af624cb","Type":"ContainerStarted","Data":"74eb84e9ca1b73a3a5ddf7a09fe0232cc5f58d8fbcc317649939db7204d3d86b"}
Oct 29 23:20:39.984989 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.984960    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:20:39.986101 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:39.986073    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk" event={"ID":"ed971fe7-998b-462e-bd88-3bb7001aa163","Type":"ContainerStarted","Data":"d80ac685cee45397d58e331f1bf1d92f13339c602296f29ac66868e3d34b3051"}
Oct 29 23:20:40.028617 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:40.028568    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft" podStartSLOduration=1.747539349 podStartE2EDuration="4.028554395s" totalImagesPullingTime="2.281015046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:20:40.027046333 +0000 UTC m=+419.722155239" watchObservedRunningTime="2026-10-29 23:20:40.028554395 +0000 UTC m=+419.723663274"
Oct 29 23:20:42.505679 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:42.505648    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:20:42.505987 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:42.505715    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:20:42.505987 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:42.505775    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:20:43.001279 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:43.001238    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk" event={"ID":"ed971fe7-998b-462e-bd88-3bb7001aa163","Type":"ContainerStarted","Data":"e4a4750fd46833e61a4624150181861c72d4677a273118ad29c580cc709137d4"}
Oct 29 23:20:43.073118 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:43.073059    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk" podStartSLOduration=1.273325741 podStartE2EDuration="4.073045294s" totalImagesPullingTime="2.799719553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:20:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:20:43.060265686 +0000 UTC m=+422.755374588" watchObservedRunningTime="2026-10-29 23:20:43.073045294 +0000 UTC m=+422.768154173"
Oct 29 23:20:43.384608 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:43.384574    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:43.389311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:43.389283    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:44.005543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:44.005497    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:44.006422 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:20:44.006401    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-557c4f746fjz9lk"
Oct 29 23:20:59.777949 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:20:59.777908    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:21:01.001251 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:01.001214    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-cdpft"
Oct 29 23:21:24.696453 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.696380    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-qd5tw"]
Oct 29 23:21:24.707258 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.707232    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw"
Oct 29 23:21:24.731876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.731853    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-h9ps4\""
Oct 29 23:21:24.743933 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.743910    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Oct 29 23:21:24.745219 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.745198    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-qd5tw"]
Oct 29 23:21:24.753504 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.753478    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"]
Oct 29 23:21:24.756827 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.756812    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:21:24.776892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.776869    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Oct 29 23:21:24.777135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.777113    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-85c2r\""
Oct 29 23:21:24.779355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.779333    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"]
Oct 29 23:21:24.782459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.782438    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f6782eb-47b3-4936-ace2-0836f4821633-cert\") pod \"kserve-controller-manager-776cc66d47-qd5tw\" (UID: \"5f6782eb-47b3-4936-ace2-0836f4821633\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw"
Oct 29 23:21:24.782540 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.782498    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwt2b\" (UniqueName: \"kubernetes.io/projected/5f6782eb-47b3-4936-ace2-0836f4821633-kube-api-access-nwt2b\") pod \"kserve-controller-manager-776cc66d47-qd5tw\" (UID: \"5f6782eb-47b3-4936-ace2-0836f4821633\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw"
Oct 29 23:21:24.787365 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.787347    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-5vwwh"]
Oct 29 23:21:24.790710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.790694    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh"
Oct 29 23:21:24.811650 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.811618    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-5vwwh"]
Oct 29 23:21:24.878989 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.878963    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Oct 29 23:21:24.878989 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.878986    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-fjc7d\""
Oct 29 23:21:24.883373 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.883347    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f6782eb-47b3-4936-ace2-0836f4821633-cert\") pod \"kserve-controller-manager-776cc66d47-qd5tw\" (UID: \"5f6782eb-47b3-4936-ace2-0836f4821633\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw"
Oct 29 23:21:24.883534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.883392    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92a890ef-2b9d-4303-9cf8-7d64189c24f8-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-t94d9\" (UID: \"92a890ef-2b9d-4303-9cf8-7d64189c24f8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:21:24.883534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.883422    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0483ae56-3819-4943-9d51-a18dd82502bc-tls-certs\") pod \"model-serving-api-75bd89fdfc-5vwwh\" (UID: \"0483ae56-3819-4943-9d51-a18dd82502bc\") " pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh"
Oct 29 23:21:24.883534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.883474    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwt2b\" (UniqueName: \"kubernetes.io/projected/5f6782eb-47b3-4936-ace2-0836f4821633-kube-api-access-nwt2b\") pod \"kserve-controller-manager-776cc66d47-qd5tw\" (UID: \"5f6782eb-47b3-4936-ace2-0836f4821633\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw"
Oct 29 23:21:24.883534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.883512    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smh84\" (UniqueName: \"kubernetes.io/projected/92a890ef-2b9d-4303-9cf8-7d64189c24f8-kube-api-access-smh84\") pod \"llmisvc-controller-manager-869d5dcc5d-t94d9\" (UID: \"92a890ef-2b9d-4303-9cf8-7d64189c24f8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:21:24.883769 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.883540    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scf4h\" (UniqueName: \"kubernetes.io/projected/0483ae56-3819-4943-9d51-a18dd82502bc-kube-api-access-scf4h\") pod \"model-serving-api-75bd89fdfc-5vwwh\" (UID: \"0483ae56-3819-4943-9d51-a18dd82502bc\") " pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh"
Oct 29 23:21:24.886255 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.886234    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f6782eb-47b3-4936-ace2-0836f4821633-cert\") pod \"kserve-controller-manager-776cc66d47-qd5tw\" (UID: \"5f6782eb-47b3-4936-ace2-0836f4821633\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw"
Oct 29 23:21:24.923385 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.923363    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwt2b\" (UniqueName: \"kubernetes.io/projected/5f6782eb-47b3-4936-ace2-0836f4821633-kube-api-access-nwt2b\") pod \"kserve-controller-manager-776cc66d47-qd5tw\" (UID: \"5f6782eb-47b3-4936-ace2-0836f4821633\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw"
Oct 29 23:21:24.984029 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.983952    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92a890ef-2b9d-4303-9cf8-7d64189c24f8-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-t94d9\" (UID: \"92a890ef-2b9d-4303-9cf8-7d64189c24f8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:21:24.984029 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.983991    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0483ae56-3819-4943-9d51-a18dd82502bc-tls-certs\") pod \"model-serving-api-75bd89fdfc-5vwwh\" (UID: \"0483ae56-3819-4943-9d51-a18dd82502bc\") " pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh"
Oct 29 23:21:24.984239 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.984043    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smh84\" (UniqueName: \"kubernetes.io/projected/92a890ef-2b9d-4303-9cf8-7d64189c24f8-kube-api-access-smh84\") pod \"llmisvc-controller-manager-869d5dcc5d-t94d9\" (UID: \"92a890ef-2b9d-4303-9cf8-7d64189c24f8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:21:24.984239 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.984072    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scf4h\" (UniqueName: \"kubernetes.io/projected/0483ae56-3819-4943-9d51-a18dd82502bc-kube-api-access-scf4h\") pod \"model-serving-api-75bd89fdfc-5vwwh\" (UID: \"0483ae56-3819-4943-9d51-a18dd82502bc\") " pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh"
Oct 29 23:21:24.986374 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.986342    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0483ae56-3819-4943-9d51-a18dd82502bc-tls-certs\") pod \"model-serving-api-75bd89fdfc-5vwwh\" (UID: \"0483ae56-3819-4943-9d51-a18dd82502bc\") " pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh"
Oct 29 23:21:24.986504 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:24.986483    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92a890ef-2b9d-4303-9cf8-7d64189c24f8-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-t94d9\" (UID: \"92a890ef-2b9d-4303-9cf8-7d64189c24f8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:21:25.007329 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:25.007305    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scf4h\" (UniqueName: \"kubernetes.io/projected/0483ae56-3819-4943-9d51-a18dd82502bc-kube-api-access-scf4h\") pod \"model-serving-api-75bd89fdfc-5vwwh\" (UID: \"0483ae56-3819-4943-9d51-a18dd82502bc\") " pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh"
Oct 29 23:21:25.007474 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:25.007448    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smh84\" (UniqueName: \"kubernetes.io/projected/92a890ef-2b9d-4303-9cf8-7d64189c24f8-kube-api-access-smh84\") pod \"llmisvc-controller-manager-869d5dcc5d-t94d9\" (UID: \"92a890ef-2b9d-4303-9cf8-7d64189c24f8\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:21:25.019138 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:25.019118    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw"
Oct 29 23:21:25.068496 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:25.068360    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:21:25.103129 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:25.103095    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh"
Oct 29 23:21:25.317010 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:25.316925    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-qd5tw"]
Oct 29 23:21:25.320059 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:21:25.320034    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f6782eb_47b3_4936_ace2_0836f4821633.slice/crio-8955705fccd2ece35b30975f5ffed5c15117cf27a4c5c3e0423dfab4fb330659 WatchSource:0}: Error finding container 8955705fccd2ece35b30975f5ffed5c15117cf27a4c5c3e0423dfab4fb330659: Status 404 returned error can't find the container with id 8955705fccd2ece35b30975f5ffed5c15117cf27a4c5c3e0423dfab4fb330659
Oct 29 23:21:25.367583 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:25.367556    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"]
Oct 29 23:21:25.369886 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:21:25.369860    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod92a890ef_2b9d_4303_9cf8_7d64189c24f8.slice/crio-a0d01dbb64c9ff9c7a9b905102d24d9eeb4dc09a1d39b7195748e5b2d7f1047d WatchSource:0}: Error finding container a0d01dbb64c9ff9c7a9b905102d24d9eeb4dc09a1d39b7195748e5b2d7f1047d: Status 404 returned error can't find the container with id a0d01dbb64c9ff9c7a9b905102d24d9eeb4dc09a1d39b7195748e5b2d7f1047d
Oct 29 23:21:25.407391 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:25.407361    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-5vwwh"]
Oct 29 23:21:25.410481 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:21:25.410455    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0483ae56_3819_4943_9d51_a18dd82502bc.slice/crio-cc883f357ddad660c0e288307e1843765f5416c5745f65bba2c2150a57aac600 WatchSource:0}: Error finding container cc883f357ddad660c0e288307e1843765f5416c5745f65bba2c2150a57aac600: Status 404 returned error can't find the container with id cc883f357ddad660c0e288307e1843765f5416c5745f65bba2c2150a57aac600
Oct 29 23:21:26.201191 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:26.201157    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh" event={"ID":"0483ae56-3819-4943-9d51-a18dd82502bc","Type":"ContainerStarted","Data":"cc883f357ddad660c0e288307e1843765f5416c5745f65bba2c2150a57aac600"}
Oct 29 23:21:26.204513 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:26.204449    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9" event={"ID":"92a890ef-2b9d-4303-9cf8-7d64189c24f8","Type":"ContainerStarted","Data":"a0d01dbb64c9ff9c7a9b905102d24d9eeb4dc09a1d39b7195748e5b2d7f1047d"}
Oct 29 23:21:26.206680 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:26.206623    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw" event={"ID":"5f6782eb-47b3-4936-ace2-0836f4821633","Type":"ContainerStarted","Data":"8955705fccd2ece35b30975f5ffed5c15117cf27a4c5c3e0423dfab4fb330659"}
Oct 29 23:21:28.218342 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:28.218303    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh" event={"ID":"0483ae56-3819-4943-9d51-a18dd82502bc","Type":"ContainerStarted","Data":"3f7b347f0b54de2bd669dcc9a5b011472104337b5a04e50f7315e8b56d7769bb"}
Oct 29 23:21:28.218836 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:28.218488    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh"
Oct 29 23:21:28.219972 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:28.219947    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9" event={"ID":"92a890ef-2b9d-4303-9cf8-7d64189c24f8","Type":"ContainerStarted","Data":"dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0"}
Oct 29 23:21:28.220116 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:28.220026    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:21:28.221438 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:28.221413    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw" event={"ID":"5f6782eb-47b3-4936-ace2-0836f4821633","Type":"ContainerStarted","Data":"ae54f054f8df4db0bb29e8b1e46340a81a986f6cb179d51940909a9d733dc7c3"}
Oct 29 23:21:28.221565 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:28.221499    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw"
Oct 29 23:21:28.259907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:28.259849    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh" podStartSLOduration=2.3491724720000002 podStartE2EDuration="4.259836842s" totalImagesPullingTime="1.91066437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:21:28.25909282 +0000 UTC m=+467.954201700" watchObservedRunningTime="2026-10-29 23:21:28.259836842 +0000 UTC m=+467.954945720"
Oct 29 23:21:28.306968 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:28.306921    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw" podStartSLOduration=1.928864396 podStartE2EDuration="4.306900485s" totalImagesPullingTime="2.378036089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:21:28.306119925 +0000 UTC m=+468.001228804" watchObservedRunningTime="2026-10-29 23:21:28.306900485 +0000 UTC m=+468.002009364"
Oct 29 23:21:28.346584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:28.346536    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9" podStartSLOduration=2.021447692 podStartE2EDuration="4.346523185s" totalImagesPullingTime="2.325075493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:21:28.345656461 +0000 UTC m=+468.040765339" watchObservedRunningTime="2026-10-29 23:21:28.346523185 +0000 UTC m=+468.041632063"
Oct 29 23:21:29.778057 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:21:29.778015    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:21:39.230486 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:39.230457    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-5vwwh"
Oct 29 23:21:59.229423 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:59.229391    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:21:59.232595 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:21:59.232574    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-qd5tw"
Oct 29 23:21:59.780442 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:21:59.780399    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:22:16.224830 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.224795    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"]
Oct 29 23:22:16.229750 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.229721    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:16.257004 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.256974    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 29 23:22:16.257004 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.256987    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-7qndx\""
Oct 29 23:22:16.257212 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.257038    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 29 23:22:16.258305 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.258282    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"]
Oct 29 23:22:16.321611 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.321582    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"]
Oct 29 23:22:16.324896 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.324877    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:16.329150 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.329124    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xth4m\" (UniqueName: \"kubernetes.io/projected/336db286-2009-4d41-9e78-4faf703c8f65-kube-api-access-xth4m\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:16.329272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.329239    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:16.329335 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.329301    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:16.342144 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.342115    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-5ltz7\""
Oct 29 23:22:16.343815 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.343795    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"]
Oct 29 23:22:16.425509 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.425475    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"]
Oct 29 23:22:16.429155 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.429136    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:16.429675 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.429651    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:16.429777 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.429690    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:16.429777 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.429751    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:16.429889 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.429816    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xth4m\" (UniqueName: \"kubernetes.io/projected/336db286-2009-4d41-9e78-4faf703c8f65-kube-api-access-xth4m\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:16.429889 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.429853    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j68z\" (UniqueName: \"kubernetes.io/projected/a1836649-ca1b-42e1-a388-6672c1bcc8a4-kube-api-access-2j68z\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:16.429993 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.429904    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:16.430119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.430097    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:16.430170 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.430161    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:16.446469 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.446446    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-smct7\""
Oct 29 23:22:16.450390 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.450366    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"]
Oct 29 23:22:16.456405 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.455520    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xth4m\" (UniqueName: \"kubernetes.io/projected/336db286-2009-4d41-9e78-4faf703c8f65-kube-api-access-xth4m\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:16.526349 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.526287    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"]
Oct 29 23:22:16.529744 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.529721    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:16.530307 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.530285    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:16.530424 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.530339    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:16.530424 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.530362    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:16.530537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.530503    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:16.530596 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.530546    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfhrt\" (UniqueName: \"kubernetes.io/projected/fac0844e-10d9-42ae-b6af-adab374ae076-kube-api-access-nfhrt\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:16.530596 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.530566    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j68z\" (UniqueName: \"kubernetes.io/projected/a1836649-ca1b-42e1-a388-6672c1bcc8a4-kube-api-access-2j68z\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:16.530723 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.530704    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:16.530862 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.530844    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:16.544165 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.544148    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:16.547253 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.547221    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-qsz62\""
Oct 29 23:22:16.549588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.549568    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"]
Oct 29 23:22:16.556196 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.556178    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j68z\" (UniqueName: \"kubernetes.io/projected/a1836649-ca1b-42e1-a388-6672c1bcc8a4-kube-api-access-2j68z\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:16.631942 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.631911    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfhrt\" (UniqueName: \"kubernetes.io/projected/fac0844e-10d9-42ae-b6af-adab374ae076-kube-api-access-nfhrt\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:16.632105 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.631963    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:16.632105 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.631984    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsgtv\" (UniqueName: \"kubernetes.io/projected/129f5944-f35e-40a9-9a8e-82fa821fb684-kube-api-access-zsgtv\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:16.632105 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.632014    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:16.632254 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.632154    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:16.632254 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.632171    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:16.632567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.632548    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:16.632567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.632561    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:16.640819 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.640797    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:16.655574 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.655549    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfhrt\" (UniqueName: \"kubernetes.io/projected/fac0844e-10d9-42ae-b6af-adab374ae076-kube-api-access-nfhrt\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:16.733189 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.733157    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:16.733189 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.733190    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsgtv\" (UniqueName: \"kubernetes.io/projected/129f5944-f35e-40a9-9a8e-82fa821fb684-kube-api-access-zsgtv\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:16.733426 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.733210    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:16.733601 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.733578    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:16.733601 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.733592    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:16.745041 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.745021    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:16.756583 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.756556    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsgtv\" (UniqueName: \"kubernetes.io/projected/129f5944-f35e-40a9-9a8e-82fa821fb684-kube-api-access-zsgtv\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:16.844626 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.844269    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"]
Oct 29 23:22:16.845019 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.844997    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:16.846452 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:22:16.846430    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336db286_2009_4d41_9e78_4faf703c8f65.slice/crio-78e9ef854148c8a2f8b158049bf2d0b9af4917eaa4076d2d35ba53aef63ce509 WatchSource:0}: Error finding container 78e9ef854148c8a2f8b158049bf2d0b9af4917eaa4076d2d35ba53aef63ce509: Status 404 returned error can't find the container with id 78e9ef854148c8a2f8b158049bf2d0b9af4917eaa4076d2d35ba53aef63ce509
Oct 29 23:22:16.938418 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:16.938396    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"]
Oct 29 23:22:16.940934 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:22:16.940891    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1836649_ca1b_42e1_a388_6672c1bcc8a4.slice/crio-a9d8fed9f057a77366a504e5fa340439cfda3be33614f92bced117363d863518 WatchSource:0}: Error finding container a9d8fed9f057a77366a504e5fa340439cfda3be33614f92bced117363d863518: Status 404 returned error can't find the container with id a9d8fed9f057a77366a504e5fa340439cfda3be33614f92bced117363d863518
Oct 29 23:22:17.040010 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.039981    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"]
Oct 29 23:22:17.043524 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:22:17.043491    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac0844e_10d9_42ae_b6af_adab374ae076.slice/crio-562f94b7519ae1278da93b048da15000c2a23ccd6ae7feac78f9fdd3f93a586b WatchSource:0}: Error finding container 562f94b7519ae1278da93b048da15000c2a23ccd6ae7feac78f9fdd3f93a586b: Status 404 returned error can't find the container with id 562f94b7519ae1278da93b048da15000c2a23ccd6ae7feac78f9fdd3f93a586b
Oct 29 23:22:17.144784 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.144724    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"]
Oct 29 23:22:17.177616 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:22:17.177585    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod129f5944_f35e_40a9_9a8e_82fa821fb684.slice/crio-ce31fbc59486b7d8f75b3f4cf517ab3ef354205ebdb2d7014ab24dd2c5953839 WatchSource:0}: Error finding container ce31fbc59486b7d8f75b3f4cf517ab3ef354205ebdb2d7014ab24dd2c5953839: Status 404 returned error can't find the container with id ce31fbc59486b7d8f75b3f4cf517ab3ef354205ebdb2d7014ab24dd2c5953839
Oct 29 23:22:17.432976 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.432943    6040 generic.go:356] "Generic (PLEG): container finished" podID="129f5944-f35e-40a9-9a8e-82fa821fb684" containerID="eaa5405cd85b0042904f443cf132bdde9c2b749718b5cf7582f4da8ee2ec1c52" exitCode=0
Oct 29 23:22:17.433401 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.433025    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5" event={"ID":"129f5944-f35e-40a9-9a8e-82fa821fb684","Type":"ContainerDied","Data":"eaa5405cd85b0042904f443cf132bdde9c2b749718b5cf7582f4da8ee2ec1c52"}
Oct 29 23:22:17.433401 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.433086    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5" event={"ID":"129f5944-f35e-40a9-9a8e-82fa821fb684","Type":"ContainerStarted","Data":"ce31fbc59486b7d8f75b3f4cf517ab3ef354205ebdb2d7014ab24dd2c5953839"}
Oct 29 23:22:17.434491 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.434468    6040 generic.go:356] "Generic (PLEG): container finished" podID="a1836649-ca1b-42e1-a388-6672c1bcc8a4" containerID="861c327dd262874e43ec953cd36084b56343019620b9cbe380622c68463a843d" exitCode=0
Oct 29 23:22:17.434597 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.434531    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w" event={"ID":"a1836649-ca1b-42e1-a388-6672c1bcc8a4","Type":"ContainerDied","Data":"861c327dd262874e43ec953cd36084b56343019620b9cbe380622c68463a843d"}
Oct 29 23:22:17.434597 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.434571    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w" event={"ID":"a1836649-ca1b-42e1-a388-6672c1bcc8a4","Type":"ContainerStarted","Data":"a9d8fed9f057a77366a504e5fa340439cfda3be33614f92bced117363d863518"}
Oct 29 23:22:17.436225 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.436206    6040 generic.go:356] "Generic (PLEG): container finished" podID="336db286-2009-4d41-9e78-4faf703c8f65" containerID="ac2def3d4d50e18b4e952087b2256cc2da526a1dd1cfff395651c47c5a45deeb" exitCode=0
Oct 29 23:22:17.436313 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.436289    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth" event={"ID":"336db286-2009-4d41-9e78-4faf703c8f65","Type":"ContainerDied","Data":"ac2def3d4d50e18b4e952087b2256cc2da526a1dd1cfff395651c47c5a45deeb"}
Oct 29 23:22:17.436384 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.436323    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth" event={"ID":"336db286-2009-4d41-9e78-4faf703c8f65","Type":"ContainerStarted","Data":"78e9ef854148c8a2f8b158049bf2d0b9af4917eaa4076d2d35ba53aef63ce509"}
Oct 29 23:22:17.437968 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.437948    6040 generic.go:356] "Generic (PLEG): container finished" podID="fac0844e-10d9-42ae-b6af-adab374ae076" containerID="d4c5201638828608ee65cfabbc2dccf94df838cb5b68b759f95298d74136cd5d" exitCode=0
Oct 29 23:22:17.438035 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.438005    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69" event={"ID":"fac0844e-10d9-42ae-b6af-adab374ae076","Type":"ContainerDied","Data":"d4c5201638828608ee65cfabbc2dccf94df838cb5b68b759f95298d74136cd5d"}
Oct 29 23:22:17.438078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:17.438056    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69" event={"ID":"fac0844e-10d9-42ae-b6af-adab374ae076","Type":"ContainerStarted","Data":"562f94b7519ae1278da93b048da15000c2a23ccd6ae7feac78f9fdd3f93a586b"}
Oct 29 23:22:18.444576 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:18.444537    6040 generic.go:356] "Generic (PLEG): container finished" podID="336db286-2009-4d41-9e78-4faf703c8f65" containerID="eec7030392618c271cdab74a82c4e85b03ad7ec2ff749958e5cf99c57226c644" exitCode=0
Oct 29 23:22:18.445027 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:18.444628    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth" event={"ID":"336db286-2009-4d41-9e78-4faf703c8f65","Type":"ContainerDied","Data":"eec7030392618c271cdab74a82c4e85b03ad7ec2ff749958e5cf99c57226c644"}
Oct 29 23:22:18.446401 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:18.446379    6040 generic.go:356] "Generic (PLEG): container finished" podID="fac0844e-10d9-42ae-b6af-adab374ae076" containerID="e07eef8d33c6a1b29f29cbd7834459f3b350a3d93372905ff1fdbe499fab7276" exitCode=0
Oct 29 23:22:18.446478 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:18.446450    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69" event={"ID":"fac0844e-10d9-42ae-b6af-adab374ae076","Type":"ContainerDied","Data":"e07eef8d33c6a1b29f29cbd7834459f3b350a3d93372905ff1fdbe499fab7276"}
Oct 29 23:22:18.448535 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:18.448513    6040 generic.go:356] "Generic (PLEG): container finished" podID="129f5944-f35e-40a9-9a8e-82fa821fb684" containerID="0b4e6d8bcced51050d391bb0a5d5a518b1db935e119929d7c81c6dd5602837f5" exitCode=0
Oct 29 23:22:18.448653 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:18.448587    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5" event={"ID":"129f5944-f35e-40a9-9a8e-82fa821fb684","Type":"ContainerDied","Data":"0b4e6d8bcced51050d391bb0a5d5a518b1db935e119929d7c81c6dd5602837f5"}
Oct 29 23:22:18.450296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:18.450279    6040 generic.go:356] "Generic (PLEG): container finished" podID="a1836649-ca1b-42e1-a388-6672c1bcc8a4" containerID="9672395a30e0d80aa4d04b30ec72c093fbeb53d8da08b95c17fefbcb1d163312" exitCode=0
Oct 29 23:22:18.450345 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:18.450309    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w" event={"ID":"a1836649-ca1b-42e1-a388-6672c1bcc8a4","Type":"ContainerDied","Data":"9672395a30e0d80aa4d04b30ec72c093fbeb53d8da08b95c17fefbcb1d163312"}
Oct 29 23:22:19.457240 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:19.457207    6040 generic.go:356] "Generic (PLEG): container finished" podID="129f5944-f35e-40a9-9a8e-82fa821fb684" containerID="08ffdb91770db421ad09275cc296b2b3ed490cd390697cda84d9115777a60470" exitCode=0
Oct 29 23:22:19.457693 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:19.457290    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5" event={"ID":"129f5944-f35e-40a9-9a8e-82fa821fb684","Type":"ContainerDied","Data":"08ffdb91770db421ad09275cc296b2b3ed490cd390697cda84d9115777a60470"}
Oct 29 23:22:19.459533 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:19.459506    6040 generic.go:356] "Generic (PLEG): container finished" podID="a1836649-ca1b-42e1-a388-6672c1bcc8a4" containerID="e82dfa2f8820b6aafe06a9f69e645471cb9853fb494880cbf3732428d542b583" exitCode=0
Oct 29 23:22:19.459654 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:19.459582    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w" event={"ID":"a1836649-ca1b-42e1-a388-6672c1bcc8a4","Type":"ContainerDied","Data":"e82dfa2f8820b6aafe06a9f69e645471cb9853fb494880cbf3732428d542b583"}
Oct 29 23:22:19.461600 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:19.461565    6040 generic.go:356] "Generic (PLEG): container finished" podID="336db286-2009-4d41-9e78-4faf703c8f65" containerID="6f773efc689e080dfcfbaa32d87015c697261465d94c81db55b2253cca3c3aee" exitCode=0
Oct 29 23:22:19.461689 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:19.461647    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth" event={"ID":"336db286-2009-4d41-9e78-4faf703c8f65","Type":"ContainerDied","Data":"6f773efc689e080dfcfbaa32d87015c697261465d94c81db55b2253cca3c3aee"}
Oct 29 23:22:19.463683 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:19.463663    6040 generic.go:356] "Generic (PLEG): container finished" podID="fac0844e-10d9-42ae-b6af-adab374ae076" containerID="a16f378d9c0c0bd6873dc3f553041ce375d954f1195e64fff30abc3f18e08aab" exitCode=0
Oct 29 23:22:19.463793 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:19.463759    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69" event={"ID":"fac0844e-10d9-42ae-b6af-adab374ae076","Type":"ContainerDied","Data":"a16f378d9c0c0bd6873dc3f553041ce375d954f1195e64fff30abc3f18e08aab"}
Oct 29 23:22:20.642700 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.642678    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:20.646307 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.646287    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:20.671032 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.671015    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:20.673862 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.673845    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:20.767612 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.767502    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-util\" (UniqueName: \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-util\") pod \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") "
Oct 29 23:22:20.767612 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.767573    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/336db286-2009-4d41-9e78-4faf703c8f65-kube-api-access-xth4m\" (UniqueName: \"kubernetes.io/projected/336db286-2009-4d41-9e78-4faf703c8f65-kube-api-access-xth4m\") pod \"336db286-2009-4d41-9e78-4faf703c8f65\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") "
Oct 29 23:22:20.767938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.767685    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-bundle\" (UniqueName: \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-bundle\") pod \"fac0844e-10d9-42ae-b6af-adab374ae076\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") "
Oct 29 23:22:20.767938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.767794    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-util\" (UniqueName: \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-util\") pod \"336db286-2009-4d41-9e78-4faf703c8f65\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") "
Oct 29 23:22:20.767938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.767835    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-bundle\" (UniqueName: \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-bundle\") pod \"336db286-2009-4d41-9e78-4faf703c8f65\" (UID: \"336db286-2009-4d41-9e78-4faf703c8f65\") "
Oct 29 23:22:20.767938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.767861    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/129f5944-f35e-40a9-9a8e-82fa821fb684-kube-api-access-zsgtv\" (UniqueName: \"kubernetes.io/projected/129f5944-f35e-40a9-9a8e-82fa821fb684-kube-api-access-zsgtv\") pod \"129f5944-f35e-40a9-9a8e-82fa821fb684\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") "
Oct 29 23:22:20.767938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.767891    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-bundle\") pod \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") "
Oct 29 23:22:20.768206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.767948    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-bundle\" (UniqueName: \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-bundle\") pod \"129f5944-f35e-40a9-9a8e-82fa821fb684\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") "
Oct 29 23:22:20.768206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.767999    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1836649-ca1b-42e1-a388-6672c1bcc8a4-kube-api-access-2j68z\" (UniqueName: \"kubernetes.io/projected/a1836649-ca1b-42e1-a388-6672c1bcc8a4-kube-api-access-2j68z\") pod \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\" (UID: \"a1836649-ca1b-42e1-a388-6672c1bcc8a4\") "
Oct 29 23:22:20.768206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.768030    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fac0844e-10d9-42ae-b6af-adab374ae076-kube-api-access-nfhrt\" (UniqueName: \"kubernetes.io/projected/fac0844e-10d9-42ae-b6af-adab374ae076-kube-api-access-nfhrt\") pod \"fac0844e-10d9-42ae-b6af-adab374ae076\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") "
Oct 29 23:22:20.768206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.768101    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-util\" (UniqueName: \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-util\") pod \"fac0844e-10d9-42ae-b6af-adab374ae076\" (UID: \"fac0844e-10d9-42ae-b6af-adab374ae076\") "
Oct 29 23:22:20.768206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.768156    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-util\" (UniqueName: \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-util\") pod \"129f5944-f35e-40a9-9a8e-82fa821fb684\" (UID: \"129f5944-f35e-40a9-9a8e-82fa821fb684\") "
Oct 29 23:22:20.770272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.768759    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-bundle" pod "fac0844e-10d9-42ae-b6af-adab374ae076" (UID: "fac0844e-10d9-42ae-b6af-adab374ae076"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:22:20.770272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.769597    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-bundle" pod "a1836649-ca1b-42e1-a388-6672c1bcc8a4" (UID: "a1836649-ca1b-42e1-a388-6672c1bcc8a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:22:20.770272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.769624    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-bundle" pod "336db286-2009-4d41-9e78-4faf703c8f65" (UID: "336db286-2009-4d41-9e78-4faf703c8f65"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:22:20.770272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.770096    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-bundle" pod "129f5944-f35e-40a9-9a8e-82fa821fb684" (UID: "129f5944-f35e-40a9-9a8e-82fa821fb684"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:22:20.773675 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.773641    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fac0844e-10d9-42ae-b6af-adab374ae076-kube-api-access-nfhrt" pod "fac0844e-10d9-42ae-b6af-adab374ae076" (UID: "fac0844e-10d9-42ae-b6af-adab374ae076"). InnerVolumeSpecName "kube-api-access-nfhrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:22:20.773792 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.773724    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/129f5944-f35e-40a9-9a8e-82fa821fb684-kube-api-access-zsgtv" pod "129f5944-f35e-40a9-9a8e-82fa821fb684" (UID: "129f5944-f35e-40a9-9a8e-82fa821fb684"). InnerVolumeSpecName "kube-api-access-zsgtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:22:20.773792 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.773764    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/336db286-2009-4d41-9e78-4faf703c8f65-kube-api-access-xth4m" pod "336db286-2009-4d41-9e78-4faf703c8f65" (UID: "336db286-2009-4d41-9e78-4faf703c8f65"). InnerVolumeSpecName "kube-api-access-xth4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:22:20.775650 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.775623    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1836649-ca1b-42e1-a388-6672c1bcc8a4-kube-api-access-2j68z" pod "a1836649-ca1b-42e1-a388-6672c1bcc8a4" (UID: "a1836649-ca1b-42e1-a388-6672c1bcc8a4"). InnerVolumeSpecName "kube-api-access-2j68z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:22:20.777123 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.777100    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-util" pod "129f5944-f35e-40a9-9a8e-82fa821fb684" (UID: "129f5944-f35e-40a9-9a8e-82fa821fb684"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:22:20.777918 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.777899    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-util" pod "a1836649-ca1b-42e1-a388-6672c1bcc8a4" (UID: "a1836649-ca1b-42e1-a388-6672c1bcc8a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:22:20.779135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.779095    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-util" pod "336db286-2009-4d41-9e78-4faf703c8f65" (UID: "336db286-2009-4d41-9e78-4faf703c8f65"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:22:20.779753 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.779717    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-util" pod "fac0844e-10d9-42ae-b6af-adab374ae076" (UID: "fac0844e-10d9-42ae-b6af-adab374ae076"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:22:20.869393 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869352    6040 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-util\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:20.869393 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869385    6040 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-util\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:20.869393 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869396    6040 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-util\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:20.869632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869407    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xth4m\" (UniqueName: \"kubernetes.io/projected/336db286-2009-4d41-9e78-4faf703c8f65-kube-api-access-xth4m\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:20.869632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869419    6040 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fac0844e-10d9-42ae-b6af-adab374ae076-bundle\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:20.869632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869433    6040 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-util\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:20.869632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869442    6040 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/336db286-2009-4d41-9e78-4faf703c8f65-bundle\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:20.869632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869452    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsgtv\" (UniqueName: \"kubernetes.io/projected/129f5944-f35e-40a9-9a8e-82fa821fb684-kube-api-access-zsgtv\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:20.869632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869463    6040 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1836649-ca1b-42e1-a388-6672c1bcc8a4-bundle\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:20.869632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869473    6040 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/129f5944-f35e-40a9-9a8e-82fa821fb684-bundle\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:20.869632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869484    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j68z\" (UniqueName: \"kubernetes.io/projected/a1836649-ca1b-42e1-a388-6672c1bcc8a4-kube-api-access-2j68z\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:20.869632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:20.869495    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfhrt\" (UniqueName: \"kubernetes.io/projected/fac0844e-10d9-42ae-b6af-adab374ae076-kube-api-access-nfhrt\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:21.474545 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.474509    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth" event={"ID":"336db286-2009-4d41-9e78-4faf703c8f65","Type":"ContainerDied","Data":"78e9ef854148c8a2f8b158049bf2d0b9af4917eaa4076d2d35ba53aef63ce509"}
Oct 29 23:22:21.474709 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.474554    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78e9ef854148c8a2f8b158049bf2d0b9af4917eaa4076d2d35ba53aef63ce509"
Oct 29 23:22:21.474709 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.474516    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hwnth"
Oct 29 23:22:21.476431 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.476411    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69"
Oct 29 23:22:21.476535 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.476480    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvsk69" event={"ID":"fac0844e-10d9-42ae-b6af-adab374ae076","Type":"ContainerDied","Data":"562f94b7519ae1278da93b048da15000c2a23ccd6ae7feac78f9fdd3f93a586b"}
Oct 29 23:22:21.476535 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.476514    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="562f94b7519ae1278da93b048da15000c2a23ccd6ae7feac78f9fdd3f93a586b"
Oct 29 23:22:21.478319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.478298    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5"
Oct 29 23:22:21.478414 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.478314    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8qjx5" event={"ID":"129f5944-f35e-40a9-9a8e-82fa821fb684","Type":"ContainerDied","Data":"ce31fbc59486b7d8f75b3f4cf517ab3ef354205ebdb2d7014ab24dd2c5953839"}
Oct 29 23:22:21.478414 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.478348    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce31fbc59486b7d8f75b3f4cf517ab3ef354205ebdb2d7014ab24dd2c5953839"
Oct 29 23:22:21.480189 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.480167    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w" event={"ID":"a1836649-ca1b-42e1-a388-6672c1bcc8a4","Type":"ContainerDied","Data":"a9d8fed9f057a77366a504e5fa340439cfda3be33614f92bced117363d863518"}
Oct 29 23:22:21.480298 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.480200    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9d8fed9f057a77366a504e5fa340439cfda3be33614f92bced117363d863518"
Oct 29 23:22:21.480298 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:21.480206    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sdl9w"
Oct 29 23:22:29.778549 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:22:29.778438    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:22:33.889030 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.888993    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl"]
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889318    6040 cpu_manager.go:414] "containerMap: removing container" podUID="fac0844e-10d9-42ae-b6af-adab374ae076" containerName="util"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889329    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fac0844e-10d9-42ae-b6af-adab374ae076" containerName="util"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889341    6040 cpu_manager.go:414] "containerMap: removing container" podUID="fac0844e-10d9-42ae-b6af-adab374ae076" containerName="extract"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889349    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fac0844e-10d9-42ae-b6af-adab374ae076" containerName="extract"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889358    6040 cpu_manager.go:414] "containerMap: removing container" podUID="336db286-2009-4d41-9e78-4faf703c8f65" containerName="extract"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889363    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="336db286-2009-4d41-9e78-4faf703c8f65" containerName="extract"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889372    6040 cpu_manager.go:414] "containerMap: removing container" podUID="a1836649-ca1b-42e1-a388-6672c1bcc8a4" containerName="extract"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889376    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1836649-ca1b-42e1-a388-6672c1bcc8a4" containerName="extract"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889386    6040 cpu_manager.go:414] "containerMap: removing container" podUID="129f5944-f35e-40a9-9a8e-82fa821fb684" containerName="pull"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889390    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="129f5944-f35e-40a9-9a8e-82fa821fb684" containerName="pull"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889396    6040 cpu_manager.go:414] "containerMap: removing container" podUID="129f5944-f35e-40a9-9a8e-82fa821fb684" containerName="extract"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889399    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="129f5944-f35e-40a9-9a8e-82fa821fb684" containerName="extract"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889409    6040 cpu_manager.go:414] "containerMap: removing container" podUID="336db286-2009-4d41-9e78-4faf703c8f65" containerName="util"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889415    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="336db286-2009-4d41-9e78-4faf703c8f65" containerName="util"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889423    6040 cpu_manager.go:414] "containerMap: removing container" podUID="a1836649-ca1b-42e1-a388-6672c1bcc8a4" containerName="util"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889430    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1836649-ca1b-42e1-a388-6672c1bcc8a4" containerName="util"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889436    6040 cpu_manager.go:414] "containerMap: removing container" podUID="129f5944-f35e-40a9-9a8e-82fa821fb684" containerName="util"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889440    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="129f5944-f35e-40a9-9a8e-82fa821fb684" containerName="util"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889447    6040 cpu_manager.go:414] "containerMap: removing container" podUID="fac0844e-10d9-42ae-b6af-adab374ae076" containerName="pull"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889451    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fac0844e-10d9-42ae-b6af-adab374ae076" containerName="pull"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889459    6040 cpu_manager.go:414] "containerMap: removing container" podUID="a1836649-ca1b-42e1-a388-6672c1bcc8a4" containerName="pull"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889463    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1836649-ca1b-42e1-a388-6672c1bcc8a4" containerName="pull"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889471    6040 cpu_manager.go:414] "containerMap: removing container" podUID="336db286-2009-4d41-9e78-4faf703c8f65" containerName="pull"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889475    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="336db286-2009-4d41-9e78-4faf703c8f65" containerName="pull"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889528    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="fac0844e-10d9-42ae-b6af-adab374ae076" containerName="extract"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889537    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="336db286-2009-4d41-9e78-4faf703c8f65" containerName="extract"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889544    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1836649-ca1b-42e1-a388-6672c1bcc8a4" containerName="extract"
Oct 29 23:22:33.889569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.889550    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="129f5944-f35e-40a9-9a8e-82fa821fb684" containerName="extract"
Oct 29 23:22:33.892544 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.892528    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl"
Oct 29 23:22:33.907770 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.907749    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 29 23:22:33.919709 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.919685    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hlhd7\""
Oct 29 23:22:33.919822 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.919762    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 29 23:22:33.919874 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.919818    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 29 23:22:33.921522 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.921497    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl"]
Oct 29 23:22:33.982069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:33.982046    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ggds\" (UniqueName: \"kubernetes.io/projected/6acc30b0-f8c4-4b4c-a625-9290959971e3-kube-api-access-8ggds\") pod \"dns-operator-controller-manager-65bfd79598-jx6jl\" (UID: \"6acc30b0-f8c4-4b4c-a625-9290959971e3\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl"
Oct 29 23:22:34.082972 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:34.082920    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ggds\" (UniqueName: \"kubernetes.io/projected/6acc30b0-f8c4-4b4c-a625-9290959971e3-kube-api-access-8ggds\") pod \"dns-operator-controller-manager-65bfd79598-jx6jl\" (UID: \"6acc30b0-f8c4-4b4c-a625-9290959971e3\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl"
Oct 29 23:22:34.105759 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:34.105712    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ggds\" (UniqueName: \"kubernetes.io/projected/6acc30b0-f8c4-4b4c-a625-9290959971e3-kube-api-access-8ggds\") pod \"dns-operator-controller-manager-65bfd79598-jx6jl\" (UID: \"6acc30b0-f8c4-4b4c-a625-9290959971e3\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl"
Oct 29 23:22:34.204850 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:34.204770    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl"
Oct 29 23:22:34.505684 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:34.505660    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl"]
Oct 29 23:22:34.508266 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:22:34.508234    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6acc30b0_f8c4_4b4c_a625_9290959971e3.slice/crio-8117a51674521c838240d6549a591b0ade419763e047cb7eed831bc71f2074f0 WatchSource:0}: Error finding container 8117a51674521c838240d6549a591b0ade419763e047cb7eed831bc71f2074f0: Status 404 returned error can't find the container with id 8117a51674521c838240d6549a591b0ade419763e047cb7eed831bc71f2074f0
Oct 29 23:22:34.538638 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:34.538606    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl" event={"ID":"6acc30b0-f8c4-4b4c-a625-9290959971e3","Type":"ContainerStarted","Data":"8117a51674521c838240d6549a591b0ade419763e047cb7eed831bc71f2074f0"}
Oct 29 23:22:37.556110 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.556076    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl" event={"ID":"6acc30b0-f8c4-4b4c-a625-9290959971e3","Type":"ContainerStarted","Data":"7534f14989e3db693c516dde3a97bdc1a4668464ea25d68f8997a7cf1d6b906a"}
Oct 29 23:22:37.556588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.556200    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl"
Oct 29 23:22:37.557172 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.557153    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"]
Oct 29 23:22:37.560542 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.560526    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"
Oct 29 23:22:37.582524 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.582500    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Oct 29 23:22:37.582524 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.582516    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tbfkq\""
Oct 29 23:22:37.582678 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.582504    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Oct 29 23:22:37.584161 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.584144    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"]
Oct 29 23:22:37.599363 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.599307    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl" podStartSLOduration=2.3755770050000002 podStartE2EDuration="4.599291288s" totalImagesPullingTime="2.223714283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:22:37.599149815 +0000 UTC m=+537.294258715" watchObservedRunningTime="2026-10-29 23:22:37.599291288 +0000 UTC m=+537.294400167"
Oct 29 23:22:37.612383 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.612349    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/95520c7b-e7d6-4257-a9a5-1f102d87fe26-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-t522s\" (UID: \"95520c7b-e7d6-4257-a9a5-1f102d87fe26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"
Oct 29 23:22:37.612493 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.612401    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ztrl\" (UniqueName: \"kubernetes.io/projected/95520c7b-e7d6-4257-a9a5-1f102d87fe26-kube-api-access-4ztrl\") pod \"kuadrant-console-plugin-66c4fb575d-t522s\" (UID: \"95520c7b-e7d6-4257-a9a5-1f102d87fe26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"
Oct 29 23:22:37.612538 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.612497    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/95520c7b-e7d6-4257-a9a5-1f102d87fe26-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-t522s\" (UID: \"95520c7b-e7d6-4257-a9a5-1f102d87fe26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"
Oct 29 23:22:37.713455 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.713423    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/95520c7b-e7d6-4257-a9a5-1f102d87fe26-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-t522s\" (UID: \"95520c7b-e7d6-4257-a9a5-1f102d87fe26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"
Oct 29 23:22:37.713610 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.713473    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ztrl\" (UniqueName: \"kubernetes.io/projected/95520c7b-e7d6-4257-a9a5-1f102d87fe26-kube-api-access-4ztrl\") pod \"kuadrant-console-plugin-66c4fb575d-t522s\" (UID: \"95520c7b-e7d6-4257-a9a5-1f102d87fe26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"
Oct 29 23:22:37.713610 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.713506    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/95520c7b-e7d6-4257-a9a5-1f102d87fe26-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-t522s\" (UID: \"95520c7b-e7d6-4257-a9a5-1f102d87fe26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"
Oct 29 23:22:37.714149 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.714126    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/95520c7b-e7d6-4257-a9a5-1f102d87fe26-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-t522s\" (UID: \"95520c7b-e7d6-4257-a9a5-1f102d87fe26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"
Oct 29 23:22:37.715873 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.715853    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/95520c7b-e7d6-4257-a9a5-1f102d87fe26-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-t522s\" (UID: \"95520c7b-e7d6-4257-a9a5-1f102d87fe26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"
Oct 29 23:22:37.741141 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.741108    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ztrl\" (UniqueName: \"kubernetes.io/projected/95520c7b-e7d6-4257-a9a5-1f102d87fe26-kube-api-access-4ztrl\") pod \"kuadrant-console-plugin-66c4fb575d-t522s\" (UID: \"95520c7b-e7d6-4257-a9a5-1f102d87fe26\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"
Oct 29 23:22:37.871780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:37.871692    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"
Oct 29 23:22:38.167425 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:38.167398    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s"]
Oct 29 23:22:38.170001 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:22:38.169971    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95520c7b_e7d6_4257_a9a5_1f102d87fe26.slice/crio-b7b230d85ce4b9b884fb1031ab6e9f25aabd899edc54187b8829c9f3771a4558 WatchSource:0}: Error finding container b7b230d85ce4b9b884fb1031ab6e9f25aabd899edc54187b8829c9f3771a4558: Status 404 returned error can't find the container with id b7b230d85ce4b9b884fb1031ab6e9f25aabd899edc54187b8829c9f3771a4558
Oct 29 23:22:38.561574 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:38.561485    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s" event={"ID":"95520c7b-e7d6-4257-a9a5-1f102d87fe26","Type":"ContainerStarted","Data":"b7b230d85ce4b9b884fb1031ab6e9f25aabd899edc54187b8829c9f3771a4558"}
Oct 29 23:22:43.587903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:43.587866    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s" event={"ID":"95520c7b-e7d6-4257-a9a5-1f102d87fe26","Type":"ContainerStarted","Data":"7076d07d5ba71a12e0c6d336d09d373123833b48cdedb8ab5ed76292bd23ce53"}
Oct 29 23:22:43.636107 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:43.636053    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-t522s" podStartSLOduration=2.176783802 podStartE2EDuration="6.636036147s" totalImagesPullingTime="4.459252345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:22:43.635430934 +0000 UTC m=+543.330539983" watchObservedRunningTime="2026-10-29 23:22:43.636036147 +0000 UTC m=+543.331145027"
Oct 29 23:22:47.244803 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.244763    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"]
Oct 29 23:22:47.248361 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.248344    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:47.261166 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.261141    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"]
Oct 29 23:22:47.270991 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.270969    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9rhfr\""
Oct 29 23:22:47.295508 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.295477    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88c289b1-eac2-45eb-a353-e5524ed57483-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-g827t\" (UID: \"88c289b1-eac2-45eb-a353-e5524ed57483\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:47.295508 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.295514    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfxmc\" (UniqueName: \"kubernetes.io/projected/88c289b1-eac2-45eb-a353-e5524ed57483-kube-api-access-nfxmc\") pod \"kuadrant-operator-controller-manager-956956bc4-g827t\" (UID: \"88c289b1-eac2-45eb-a353-e5524ed57483\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:47.396927 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.396886    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88c289b1-eac2-45eb-a353-e5524ed57483-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-g827t\" (UID: \"88c289b1-eac2-45eb-a353-e5524ed57483\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:47.396927 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.396933    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfxmc\" (UniqueName: \"kubernetes.io/projected/88c289b1-eac2-45eb-a353-e5524ed57483-kube-api-access-nfxmc\") pod \"kuadrant-operator-controller-manager-956956bc4-g827t\" (UID: \"88c289b1-eac2-45eb-a353-e5524ed57483\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:47.397270 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.397250    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88c289b1-eac2-45eb-a353-e5524ed57483-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-g827t\" (UID: \"88c289b1-eac2-45eb-a353-e5524ed57483\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:47.420682 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.420649    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfxmc\" (UniqueName: \"kubernetes.io/projected/88c289b1-eac2-45eb-a353-e5524ed57483-kube-api-access-nfxmc\") pod \"kuadrant-operator-controller-manager-956956bc4-g827t\" (UID: \"88c289b1-eac2-45eb-a353-e5524ed57483\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:47.560997 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.560917    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:47.885399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:47.885374    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"]
Oct 29 23:22:47.887479 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:22:47.887452    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c289b1_eac2_45eb_a353_e5524ed57483.slice/crio-2baba5976dbb248ba4867d86e7e8124389430fab01542df56fbf60866380a5bf WatchSource:0}: Error finding container 2baba5976dbb248ba4867d86e7e8124389430fab01542df56fbf60866380a5bf: Status 404 returned error can't find the container with id 2baba5976dbb248ba4867d86e7e8124389430fab01542df56fbf60866380a5bf
Oct 29 23:22:48.564587 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:48.564548    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-jx6jl"
Oct 29 23:22:48.613034 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:48.612997    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t" event={"ID":"88c289b1-eac2-45eb-a353-e5524ed57483","Type":"ContainerStarted","Data":"2baba5976dbb248ba4867d86e7e8124389430fab01542df56fbf60866380a5bf"}
Oct 29 23:22:49.133682 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.133649    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"]
Oct 29 23:22:49.136358 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.136326    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"]
Oct 29 23:22:49.150846 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.150817    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"]
Oct 29 23:22:49.154802 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.154778    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:22:49.181908 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.181876    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"]
Oct 29 23:22:49.213655 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.213620    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89874\" (UniqueName: \"kubernetes.io/projected/83d114e8-da94-4892-9372-e28f577fc743-kube-api-access-89874\") pod \"kuadrant-operator-controller-manager-956956bc4-qmpcb\" (UID: \"83d114e8-da94-4892-9372-e28f577fc743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:22:49.213855 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.213784    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83d114e8-da94-4892-9372-e28f577fc743-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-qmpcb\" (UID: \"83d114e8-da94-4892-9372-e28f577fc743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:22:49.314869 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.314833    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83d114e8-da94-4892-9372-e28f577fc743-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-qmpcb\" (UID: \"83d114e8-da94-4892-9372-e28f577fc743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:22:49.315038 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.314936    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89874\" (UniqueName: \"kubernetes.io/projected/83d114e8-da94-4892-9372-e28f577fc743-kube-api-access-89874\") pod \"kuadrant-operator-controller-manager-956956bc4-qmpcb\" (UID: \"83d114e8-da94-4892-9372-e28f577fc743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:22:49.315315 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.315288    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83d114e8-da94-4892-9372-e28f577fc743-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-qmpcb\" (UID: \"83d114e8-da94-4892-9372-e28f577fc743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:22:49.344992 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.344963    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89874\" (UniqueName: \"kubernetes.io/projected/83d114e8-da94-4892-9372-e28f577fc743-kube-api-access-89874\") pod \"kuadrant-operator-controller-manager-956956bc4-qmpcb\" (UID: \"83d114e8-da94-4892-9372-e28f577fc743\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:22:49.471269 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.471191    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:22:49.801554 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:49.801405    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"]
Oct 29 23:22:50.626255 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:50.626158    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb" event={"ID":"83d114e8-da94-4892-9372-e28f577fc743","Type":"ContainerStarted","Data":"c1a21c3bd71f04995500c80ac784c0bec4b1dd9f0c65316ff673a0e9dc993c9c"}
Oct 29 23:22:52.640414 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:52.640375    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb" event={"ID":"83d114e8-da94-4892-9372-e28f577fc743","Type":"ContainerStarted","Data":"75061924b64762f5d48038754b28a79f84838cee9fd9454f9cfd8e51379546a1"}
Oct 29 23:22:52.640888 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:52.640448    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:22:52.642071 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:52.642049    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t" podUID="88c289b1-eac2-45eb-a353-e5524ed57483" containerName="manager" containerID="cri-o://412a369c82d1682c872315cfe44214fa781a431840b3c866b62a38ed4ea9b914" gracePeriod=2
Oct 29 23:22:52.685671 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:52.685623    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb" podStartSLOduration=1.4760099549999999 podStartE2EDuration="3.685610404s" totalImagesPullingTime="2.209600449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:22:52.68358748 +0000 UTC m=+552.378696361" watchObservedRunningTime="2026-10-29 23:22:52.685610404 +0000 UTC m=+552.380719282"
Oct 29 23:22:52.894199 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:52.894142    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:52.907562 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:22:52.907526    6040 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-956956bc4-g827t\" is forbidden: User \"system:node:ip-10-0-138-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-189.ec2.internal' and this object" podUID="88c289b1-eac2-45eb-a353-e5524ed57483" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:52.946627 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:52.946600    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88c289b1-eac2-45eb-a353-e5524ed57483-kube-api-access-nfxmc\" (UniqueName: \"kubernetes.io/projected/88c289b1-eac2-45eb-a353-e5524ed57483-kube-api-access-nfxmc\") pod \"88c289b1-eac2-45eb-a353-e5524ed57483\" (UID: \"88c289b1-eac2-45eb-a353-e5524ed57483\") "
Oct 29 23:22:52.946797 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:52.946698    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88c289b1-eac2-45eb-a353-e5524ed57483-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88c289b1-eac2-45eb-a353-e5524ed57483-extensions-socket-volume\") pod \"88c289b1-eac2-45eb-a353-e5524ed57483\" (UID: \"88c289b1-eac2-45eb-a353-e5524ed57483\") "
Oct 29 23:22:52.946998 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:52.946976    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88c289b1-eac2-45eb-a353-e5524ed57483-extensions-socket-volume" pod "88c289b1-eac2-45eb-a353-e5524ed57483" (UID: "88c289b1-eac2-45eb-a353-e5524ed57483"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:22:52.948814 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:52.948781    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88c289b1-eac2-45eb-a353-e5524ed57483-kube-api-access-nfxmc" pod "88c289b1-eac2-45eb-a353-e5524ed57483" (UID: "88c289b1-eac2-45eb-a353-e5524ed57483"). InnerVolumeSpecName "kube-api-access-nfxmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:22:53.048069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:53.048038    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfxmc\" (UniqueName: \"kubernetes.io/projected/88c289b1-eac2-45eb-a353-e5524ed57483-kube-api-access-nfxmc\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:53.048069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:53.048065    6040 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88c289b1-eac2-45eb-a353-e5524ed57483-extensions-socket-volume\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:22:53.647793 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:53.647759    6040 generic.go:356] "Generic (PLEG): container finished" podID="88c289b1-eac2-45eb-a353-e5524ed57483" containerID="412a369c82d1682c872315cfe44214fa781a431840b3c866b62a38ed4ea9b914" exitCode=2
Oct 29 23:22:53.648239 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:53.647802    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:53.648239 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:53.647832    6040 scope.go:122] "RemoveContainer" containerID="412a369c82d1682c872315cfe44214fa781a431840b3c866b62a38ed4ea9b914"
Oct 29 23:22:53.656383 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:53.656361    6040 scope.go:122] "RemoveContainer" containerID="412a369c82d1682c872315cfe44214fa781a431840b3c866b62a38ed4ea9b914"
Oct 29 23:22:53.656647 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:22:53.656628    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"412a369c82d1682c872315cfe44214fa781a431840b3c866b62a38ed4ea9b914\": container with ID starting with 412a369c82d1682c872315cfe44214fa781a431840b3c866b62a38ed4ea9b914 not found: ID does not exist" containerID="412a369c82d1682c872315cfe44214fa781a431840b3c866b62a38ed4ea9b914"
Oct 29 23:22:53.656703 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:53.656658    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"412a369c82d1682c872315cfe44214fa781a431840b3c866b62a38ed4ea9b914"} err="failed to get container status \"412a369c82d1682c872315cfe44214fa781a431840b3c866b62a38ed4ea9b914\": rpc error: code = NotFound desc = could not find container \"412a369c82d1682c872315cfe44214fa781a431840b3c866b62a38ed4ea9b914\": container with ID starting with 412a369c82d1682c872315cfe44214fa781a431840b3c866b62a38ed4ea9b914 not found: ID does not exist"
Oct 29 23:22:53.661300 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:22:53.661279    6040 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-956956bc4-g827t\" is forbidden: User \"system:node:ip-10-0-138-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-189.ec2.internal' and this object" podUID="88c289b1-eac2-45eb-a353-e5524ed57483" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:53.674512 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:22:53.674484    6040 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-956956bc4-g827t\" is forbidden: User \"system:node:ip-10-0-138-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-189.ec2.internal' and this object" podUID="88c289b1-eac2-45eb-a353-e5524ed57483" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-g827t"
Oct 29 23:22:54.423679 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:22:54.423646    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88c289b1-eac2-45eb-a353-e5524ed57483" path="/var/lib/kubelet/pods/88c289b1-eac2-45eb-a353-e5524ed57483/volumes"
Oct 29 23:22:59.779077 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:22:59.779034    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:23:03.650644 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:03.650612    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:23:16.591448 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:16.591406    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"]
Oct 29 23:23:16.591985 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:16.591620    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb" podUID="83d114e8-da94-4892-9372-e28f577fc743" containerName="manager" containerID="cri-o://75061924b64762f5d48038754b28a79f84838cee9fd9454f9cfd8e51379546a1" gracePeriod=10
Oct 29 23:23:16.762101 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:16.760958    6040 generic.go:356] "Generic (PLEG): container finished" podID="83d114e8-da94-4892-9372-e28f577fc743" containerID="75061924b64762f5d48038754b28a79f84838cee9fd9454f9cfd8e51379546a1" exitCode=0
Oct 29 23:23:16.762101 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:16.761105    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb" event={"ID":"83d114e8-da94-4892-9372-e28f577fc743","Type":"ContainerDied","Data":"75061924b64762f5d48038754b28a79f84838cee9fd9454f9cfd8e51379546a1"}
Oct 29 23:23:16.842834 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:16.842765    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:23:16.961647 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:16.961601    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83d114e8-da94-4892-9372-e28f577fc743-kube-api-access-89874\" (UniqueName: \"kubernetes.io/projected/83d114e8-da94-4892-9372-e28f577fc743-kube-api-access-89874\") pod \"83d114e8-da94-4892-9372-e28f577fc743\" (UID: \"83d114e8-da94-4892-9372-e28f577fc743\") "
Oct 29 23:23:16.961839 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:16.961727    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83d114e8-da94-4892-9372-e28f577fc743-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83d114e8-da94-4892-9372-e28f577fc743-extensions-socket-volume\") pod \"83d114e8-da94-4892-9372-e28f577fc743\" (UID: \"83d114e8-da94-4892-9372-e28f577fc743\") "
Oct 29 23:23:16.962196 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:16.962172    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83d114e8-da94-4892-9372-e28f577fc743-extensions-socket-volume" pod "83d114e8-da94-4892-9372-e28f577fc743" (UID: "83d114e8-da94-4892-9372-e28f577fc743"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:23:16.963832 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:16.963797    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83d114e8-da94-4892-9372-e28f577fc743-kube-api-access-89874" pod "83d114e8-da94-4892-9372-e28f577fc743" (UID: "83d114e8-da94-4892-9372-e28f577fc743"). InnerVolumeSpecName "kube-api-access-89874". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:23:17.062554 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:17.062494    6040 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83d114e8-da94-4892-9372-e28f577fc743-extensions-socket-volume\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:23:17.062554 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:17.062546    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89874\" (UniqueName: \"kubernetes.io/projected/83d114e8-da94-4892-9372-e28f577fc743-kube-api-access-89874\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:23:17.766943 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:17.766910    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb" event={"ID":"83d114e8-da94-4892-9372-e28f577fc743","Type":"ContainerDied","Data":"c1a21c3bd71f04995500c80ac784c0bec4b1dd9f0c65316ff673a0e9dc993c9c"}
Oct 29 23:23:17.766943 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:17.766935    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"
Oct 29 23:23:17.767457 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:17.766972    6040 scope.go:122] "RemoveContainer" containerID="75061924b64762f5d48038754b28a79f84838cee9fd9454f9cfd8e51379546a1"
Oct 29 23:23:17.833670 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:17.833638    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"]
Oct 29 23:23:17.845427 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:17.845393    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-qmpcb"]
Oct 29 23:23:18.423374 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:18.423340    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83d114e8-da94-4892-9372-e28f577fc743" path="/var/lib/kubelet/pods/83d114e8-da94-4892-9372-e28f577fc743/volumes"
Oct 29 23:23:20.869872 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.869837    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"]
Oct 29 23:23:20.870258 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.870180    6040 cpu_manager.go:414] "containerMap: removing container" podUID="88c289b1-eac2-45eb-a353-e5524ed57483" containerName="manager"
Oct 29 23:23:20.870258 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.870195    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88c289b1-eac2-45eb-a353-e5524ed57483" containerName="manager"
Oct 29 23:23:20.870258 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.870221    6040 cpu_manager.go:414] "containerMap: removing container" podUID="83d114e8-da94-4892-9372-e28f577fc743" containerName="manager"
Oct 29 23:23:20.870258 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.870226    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d114e8-da94-4892-9372-e28f577fc743" containerName="manager"
Oct 29 23:23:20.870435 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.870295    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="88c289b1-eac2-45eb-a353-e5524ed57483" containerName="manager"
Oct 29 23:23:20.870435 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.870306    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d114e8-da94-4892-9372-e28f577fc743" containerName="manager"
Oct 29 23:23:20.875254 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.875233    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.893867 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.893843    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.894001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.893872    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9lvv\" (UniqueName: \"kubernetes.io/projected/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-kube-api-access-x9lvv\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.894001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.893897    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.894001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.893955    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.894001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.893979    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.894217 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.894006    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.894217 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.894025    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.894217 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.894073    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.894217 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.894140    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.903072 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.903049    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"]
Oct 29 23:23:20.938416 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.938384    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-f5fmz\""
Oct 29 23:23:20.994876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.994847    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.994876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.994878    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9lvv\" (UniqueName: \"kubernetes.io/projected/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-kube-api-access-x9lvv\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995090 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.994931    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995090 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.994956    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995090 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.994981    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995090 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.995024    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.995146    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.995192    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.995220    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.995285    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.995313    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.995373    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.995401    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.995843 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.995814    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.997218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.997196    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:20.997341 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:20.997324    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:21.018223 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:21.018199    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:21.018340 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:21.018317    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9lvv\" (UniqueName: \"kubernetes.io/projected/07f389b2-8dd8-4149-ad51-bacbf9f1adbf-kube-api-access-x9lvv\") pod \"maas-default-gateway-openshift-default-746ff8f49f-dh2pt\" (UID: \"07f389b2-8dd8-4149-ad51-bacbf9f1adbf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:21.187428 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:21.187392    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:21.531864 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:21.531782    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"]
Oct 29 23:23:21.534976 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:23:21.534946    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f389b2_8dd8_4149_ad51_bacbf9f1adbf.slice/crio-929f60659e29abd688463a58cb2c1a9cc9e095e80d6ba4eef28fcc6df981c0a4 WatchSource:0}: Error finding container 929f60659e29abd688463a58cb2c1a9cc9e095e80d6ba4eef28fcc6df981c0a4: Status 404 returned error can't find the container with id 929f60659e29abd688463a58cb2c1a9cc9e095e80d6ba4eef28fcc6df981c0a4
Oct 29 23:23:21.536859 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:21.536827    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:23:21.536930 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:21.536891    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:23:21.536976 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:21.536932    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:23:21.790391 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:21.790304    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" event={"ID":"07f389b2-8dd8-4149-ad51-bacbf9f1adbf","Type":"ContainerStarted","Data":"ded386ea58256e3509860c9e6db619e35782d259dcdcff06a85874918183cba5"}
Oct 29 23:23:21.790391 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:21.790368    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" event={"ID":"07f389b2-8dd8-4149-ad51-bacbf9f1adbf","Type":"ContainerStarted","Data":"929f60659e29abd688463a58cb2c1a9cc9e095e80d6ba4eef28fcc6df981c0a4"}
Oct 29 23:23:21.835734 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:21.835642    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" podStartSLOduration=1.835624854 podStartE2EDuration="1.835624854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:23:21.834294783 +0000 UTC m=+581.529403663" watchObservedRunningTime="2026-10-29 23:23:21.835624854 +0000 UTC m=+581.530733736"
Oct 29 23:23:22.188117 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:22.188079    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:22.192548 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:22.192525    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:22.794657 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:22.794625    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:22.795638 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:22.795618    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:23:29.778778 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:23:29.778714    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:23:36.092702 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.092663    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:23:36.096168 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.096149    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"
Oct 29 23:23:36.114346 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.114311    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:23:36.124037 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.124007    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 29 23:23:36.179687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.179649    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:23:36.224361 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.224331    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/861c82d8-daf1-46e1-bc2f-a4a3e6b72262-config-file\") pod \"limitador-limitador-7c9b87d574-2mz5s\" (UID: \"861c82d8-daf1-46e1-bc2f-a4a3e6b72262\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"
Oct 29 23:23:36.224462 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.224391    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkrzg\" (UniqueName: \"kubernetes.io/projected/861c82d8-daf1-46e1-bc2f-a4a3e6b72262-kube-api-access-lkrzg\") pod \"limitador-limitador-7c9b87d574-2mz5s\" (UID: \"861c82d8-daf1-46e1-bc2f-a4a3e6b72262\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"
Oct 29 23:23:36.325176 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.325146    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/861c82d8-daf1-46e1-bc2f-a4a3e6b72262-config-file\") pod \"limitador-limitador-7c9b87d574-2mz5s\" (UID: \"861c82d8-daf1-46e1-bc2f-a4a3e6b72262\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"
Oct 29 23:23:36.325334 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.325211    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkrzg\" (UniqueName: \"kubernetes.io/projected/861c82d8-daf1-46e1-bc2f-a4a3e6b72262-kube-api-access-lkrzg\") pod \"limitador-limitador-7c9b87d574-2mz5s\" (UID: \"861c82d8-daf1-46e1-bc2f-a4a3e6b72262\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"
Oct 29 23:23:36.325786 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.325768    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/861c82d8-daf1-46e1-bc2f-a4a3e6b72262-config-file\") pod \"limitador-limitador-7c9b87d574-2mz5s\" (UID: \"861c82d8-daf1-46e1-bc2f-a4a3e6b72262\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"
Oct 29 23:23:36.348204 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.348145    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkrzg\" (UniqueName: \"kubernetes.io/projected/861c82d8-daf1-46e1-bc2f-a4a3e6b72262-kube-api-access-lkrzg\") pod \"limitador-limitador-7c9b87d574-2mz5s\" (UID: \"861c82d8-daf1-46e1-bc2f-a4a3e6b72262\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"
Oct 29 23:23:36.409503 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.409467    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"
Oct 29 23:23:36.705571 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.705540    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:23:36.707523 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:23:36.707494    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861c82d8_daf1_46e1_bc2f_a4a3e6b72262.slice/crio-faf5cee3cc967d5313cefc36b5ec9099b237a5f518c5a0400fbc083bf063d1ff WatchSource:0}: Error finding container faf5cee3cc967d5313cefc36b5ec9099b237a5f518c5a0400fbc083bf063d1ff: Status 404 returned error can't find the container with id faf5cee3cc967d5313cefc36b5ec9099b237a5f518c5a0400fbc083bf063d1ff
Oct 29 23:23:36.863321 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:36.863285    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s" event={"ID":"861c82d8-daf1-46e1-bc2f-a4a3e6b72262","Type":"ContainerStarted","Data":"faf5cee3cc967d5313cefc36b5ec9099b237a5f518c5a0400fbc083bf063d1ff"}
Oct 29 23:23:37.871598 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:37.871516    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s" event={"ID":"861c82d8-daf1-46e1-bc2f-a4a3e6b72262","Type":"ContainerStarted","Data":"e3c9a4b247446799ceb9bba4539749216620f14c4b1a1815d7e290c79a22a02c"}
Oct 29 23:23:37.872005 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:37.871630    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"
Oct 29 23:23:37.914315 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:37.914245    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s" podStartSLOduration=1.145084206 podStartE2EDuration="1.914224507s" totalImagesPullingTime="769.140301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:23:37.912003634 +0000 UTC m=+597.607112513" watchObservedRunningTime="2026-10-29 23:23:37.914224507 +0000 UTC m=+597.609333388"
Oct 29 23:23:48.877713 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:23:48.877685    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"
Oct 29 23:23:59.778900 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:23:59.778859    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:24:29.779606 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:24:29.779570    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:24:59.096935 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:24:59.096898    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:24:59.779295 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:24:59.779261    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:25:07.100725 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:07.100690    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"]
Oct 29 23:25:07.104195 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:07.104165    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"
Oct 29 23:25:07.124461 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:07.124433    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"]
Oct 29 23:25:07.296634 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:07.296598    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e39453e6-07f8-40ba-9622-64e5e1c70fe7-cert\") pod \"llmisvc-controller-manager-6466c554d8-jhwdt\" (UID: \"e39453e6-07f8-40ba-9622-64e5e1c70fe7\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"
Oct 29 23:25:07.296831 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:07.296653    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbvxm\" (UniqueName: \"kubernetes.io/projected/e39453e6-07f8-40ba-9622-64e5e1c70fe7-kube-api-access-bbvxm\") pod \"llmisvc-controller-manager-6466c554d8-jhwdt\" (UID: \"e39453e6-07f8-40ba-9622-64e5e1c70fe7\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"
Oct 29 23:25:07.397939 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:07.397899    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbvxm\" (UniqueName: \"kubernetes.io/projected/e39453e6-07f8-40ba-9622-64e5e1c70fe7-kube-api-access-bbvxm\") pod \"llmisvc-controller-manager-6466c554d8-jhwdt\" (UID: \"e39453e6-07f8-40ba-9622-64e5e1c70fe7\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"
Oct 29 23:25:07.398098 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:07.398063    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e39453e6-07f8-40ba-9622-64e5e1c70fe7-cert\") pod \"llmisvc-controller-manager-6466c554d8-jhwdt\" (UID: \"e39453e6-07f8-40ba-9622-64e5e1c70fe7\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"
Oct 29 23:25:07.400768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:07.400728    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e39453e6-07f8-40ba-9622-64e5e1c70fe7-cert\") pod \"llmisvc-controller-manager-6466c554d8-jhwdt\" (UID: \"e39453e6-07f8-40ba-9622-64e5e1c70fe7\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"
Oct 29 23:25:07.421486 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:07.421456    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbvxm\" (UniqueName: \"kubernetes.io/projected/e39453e6-07f8-40ba-9622-64e5e1c70fe7-kube-api-access-bbvxm\") pod \"llmisvc-controller-manager-6466c554d8-jhwdt\" (UID: \"e39453e6-07f8-40ba-9622-64e5e1c70fe7\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"
Oct 29 23:25:07.717203 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:07.717120    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"
Oct 29 23:25:08.017967 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:08.017942    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"]
Oct 29 23:25:08.020380 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:25:08.020343    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39453e6_07f8_40ba_9622_64e5e1c70fe7.slice/crio-da02593c2e93e5b5fca502252724bd0daa92b73e96f191355f9027a06be10ed3 WatchSource:0}: Error finding container da02593c2e93e5b5fca502252724bd0daa92b73e96f191355f9027a06be10ed3: Status 404 returned error can't find the container with id da02593c2e93e5b5fca502252724bd0daa92b73e96f191355f9027a06be10ed3
Oct 29 23:25:08.022221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:08.022201    6040 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 29 23:25:08.291066 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:08.290125    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-dh2pt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": read tcp 10.133.0.2:35260->10.133.0.39:15021: read: connection reset by peer" start-of-body=
Oct 29 23:25:08.291066 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:08.290187    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" podUID="07f389b2-8dd8-4149-ad51-bacbf9f1adbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": read tcp 10.133.0.2:35260->10.133.0.39:15021: read: connection reset by peer"
Oct 29 23:25:08.317763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:08.317715    6040 generic.go:356] "Generic (PLEG): container finished" podID="07f389b2-8dd8-4149-ad51-bacbf9f1adbf" containerID="ded386ea58256e3509860c9e6db619e35782d259dcdcff06a85874918183cba5" exitCode=0
Oct 29 23:25:08.317925 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:08.317767    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" event={"ID":"07f389b2-8dd8-4149-ad51-bacbf9f1adbf","Type":"ContainerDied","Data":"ded386ea58256e3509860c9e6db619e35782d259dcdcff06a85874918183cba5"}
Oct 29 23:25:08.318207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:08.318191    6040 scope.go:122] "RemoveContainer" containerID="ded386ea58256e3509860c9e6db619e35782d259dcdcff06a85874918183cba5"
Oct 29 23:25:08.319087 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:08.319066    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt" event={"ID":"e39453e6-07f8-40ba-9622-64e5e1c70fe7","Type":"ContainerStarted","Data":"da02593c2e93e5b5fca502252724bd0daa92b73e96f191355f9027a06be10ed3"}
Oct 29 23:25:08.319274 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:08.319246    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:25:08.319347 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:08.319332    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:25:08.319404 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:08.319381    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:25:09.332647 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:09.332611    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" event={"ID":"07f389b2-8dd8-4149-ad51-bacbf9f1adbf","Type":"ContainerStarted","Data":"418f0c831b457b32c795bdfa02c96bc4067bdda3e44565939bd480ae28eb037c"}
Oct 29 23:25:09.334148 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:09.334122    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt" event={"ID":"e39453e6-07f8-40ba-9622-64e5e1c70fe7","Type":"ContainerStarted","Data":"449b68d1c1c23a10589c731f8405cb871defe7058d5f2372dab2bc4eea0369d0"}
Oct 29 23:25:09.334269 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:09.334175    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"
Oct 29 23:25:09.453427 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:09.453377    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt" podStartSLOduration=2.134935435 podStartE2EDuration="2.453363629s" totalImagesPullingTime="318.428194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:25:09.451584298 +0000 UTC m=+689.146693180" watchObservedRunningTime="2026-10-29 23:25:09.453363629 +0000 UTC m=+689.148472507"
Oct 29 23:25:10.188505 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:10.188469    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:25:10.188812 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:10.188791    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-dh2pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 29 23:25:10.188871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:10.188841    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" podUID="07f389b2-8dd8-4149-ad51-bacbf9f1adbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 29 23:25:11.188194 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:11.188158    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-dh2pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 29 23:25:11.188568 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:11.188211    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" podUID="07f389b2-8dd8-4149-ad51-bacbf9f1adbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 29 23:25:12.188673 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:12.188632    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-dh2pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 29 23:25:12.189089 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:12.188691    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" podUID="07f389b2-8dd8-4149-ad51-bacbf9f1adbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 29 23:25:13.187772 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:13.187721    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-dh2pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 29 23:25:13.187945 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:13.187789    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" podUID="07f389b2-8dd8-4149-ad51-bacbf9f1adbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 29 23:25:14.188786 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:14.188711    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-dh2pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 29 23:25:14.189242 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:14.188816    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" podUID="07f389b2-8dd8-4149-ad51-bacbf9f1adbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 29 23:25:15.191439 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:15.191398    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:25:15.191873 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:15.191691    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:25:15.192269 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:15.192254    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt"
Oct 29 23:25:29.778762 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:25:29.778700    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:25:40.341479 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:40.341450    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jhwdt"
Oct 29 23:25:40.416521 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:40.416483    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"]
Oct 29 23:25:40.416726 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:40.416699    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9" podUID="92a890ef-2b9d-4303-9cf8-7d64189c24f8" containerName="manager" containerID="cri-o://dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0" gracePeriod=30
Oct 29 23:25:40.664099 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:40.664072    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:25:40.778994 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:40.778945    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92a890ef-2b9d-4303-9cf8-7d64189c24f8-kube-api-access-smh84\" (UniqueName: \"kubernetes.io/projected/92a890ef-2b9d-4303-9cf8-7d64189c24f8-kube-api-access-smh84\") pod \"92a890ef-2b9d-4303-9cf8-7d64189c24f8\" (UID: \"92a890ef-2b9d-4303-9cf8-7d64189c24f8\") "
Oct 29 23:25:40.779170 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:40.779027    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92a890ef-2b9d-4303-9cf8-7d64189c24f8-cert\" (UniqueName: \"kubernetes.io/secret/92a890ef-2b9d-4303-9cf8-7d64189c24f8-cert\") pod \"92a890ef-2b9d-4303-9cf8-7d64189c24f8\" (UID: \"92a890ef-2b9d-4303-9cf8-7d64189c24f8\") "
Oct 29 23:25:40.781157 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:40.781128    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92a890ef-2b9d-4303-9cf8-7d64189c24f8-cert" pod "92a890ef-2b9d-4303-9cf8-7d64189c24f8" (UID: "92a890ef-2b9d-4303-9cf8-7d64189c24f8"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:25:40.781157 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:40.781133    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92a890ef-2b9d-4303-9cf8-7d64189c24f8-kube-api-access-smh84" pod "92a890ef-2b9d-4303-9cf8-7d64189c24f8" (UID: "92a890ef-2b9d-4303-9cf8-7d64189c24f8"). InnerVolumeSpecName "kube-api-access-smh84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:25:40.880146 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:40.880094    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smh84\" (UniqueName: \"kubernetes.io/projected/92a890ef-2b9d-4303-9cf8-7d64189c24f8-kube-api-access-smh84\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:25:40.880146 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:40.880140    6040 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/92a890ef-2b9d-4303-9cf8-7d64189c24f8-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:25:41.487098 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:41.487064    6040 generic.go:356] "Generic (PLEG): container finished" podID="92a890ef-2b9d-4303-9cf8-7d64189c24f8" containerID="dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0" exitCode=0
Oct 29 23:25:41.487590 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:41.487111    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"
Oct 29 23:25:41.487590 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:41.487114    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9" event={"ID":"92a890ef-2b9d-4303-9cf8-7d64189c24f8","Type":"ContainerDied","Data":"dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0"}
Oct 29 23:25:41.487590 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:41.487235    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9" event={"ID":"92a890ef-2b9d-4303-9cf8-7d64189c24f8","Type":"ContainerDied","Data":"a0d01dbb64c9ff9c7a9b905102d24d9eeb4dc09a1d39b7195748e5b2d7f1047d"}
Oct 29 23:25:41.487590 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:41.487264    6040 scope.go:122] "RemoveContainer" containerID="dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0"
Oct 29 23:25:41.496279 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:41.496266    6040 scope.go:122] "RemoveContainer" containerID="dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0"
Oct 29 23:25:41.496538 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:25:41.496520    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0\": container with ID starting with dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0 not found: ID does not exist" containerID="dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0"
Oct 29 23:25:41.496624 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:41.496545    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0"} err="failed to get container status \"dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0\": rpc error: code = NotFound desc = could not find container \"dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0\": container with ID starting with dceb6f3a3addca839a525861659e8ea55f8d337037f3798910f2e877bc6b6ca0 not found: ID does not exist"
Oct 29 23:25:41.546603 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:41.546568    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"]
Oct 29 23:25:41.553497 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:41.553473    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-t94d9"]
Oct 29 23:25:42.423471 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:42.423436    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92a890ef-2b9d-4303-9cf8-7d64189c24f8" path="/var/lib/kubelet/pods/92a890ef-2b9d-4303-9cf8-7d64189c24f8/volumes"
Oct 29 23:25:59.175190 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:25:59.175110    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:25:59.783892 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:25:59.783852    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:26:03.614016 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:03.613980    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:26:09.475385 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:09.475348    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:26:16.666541 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:16.666505    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:26:29.780732 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:26:29.780685    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:26:51.993006 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:51.992971    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"]
Oct 29 23:26:51.993507 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:51.993372    6040 cpu_manager.go:414] "containerMap: removing container" podUID="92a890ef-2b9d-4303-9cf8-7d64189c24f8" containerName="manager"
Oct 29 23:26:51.993507 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:51.993389    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92a890ef-2b9d-4303-9cf8-7d64189c24f8" containerName="manager"
Oct 29 23:26:51.993507 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:51.993474    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="92a890ef-2b9d-4303-9cf8-7d64189c24f8" containerName="manager"
Oct 29 23:26:51.997868 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:51.997847    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.013918 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.013893    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"]
Oct 29 23:26:52.017199 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.017182    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.030102 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.030075    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"]
Oct 29 23:26:52.044808 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.044774    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"]
Oct 29 23:26:52.130311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130271    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.130311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130309    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5tzm\" (UniqueName: \"kubernetes.io/projected/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-kube-api-access-w5tzm\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.130533 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130327    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.130533 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130380    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.130533 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130409    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.130533 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130474    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41115057-f034-42e6-96c0-1b398529a14c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.130533 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130500    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41115057-f034-42e6-96c0-1b398529a14c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.130533 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130522    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.130728 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130598    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.130728 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130624    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.130728 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130644    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.130728 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130664    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.130728 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130727    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.130904 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130781    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.130904 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130834    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.130904 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130864    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.130904 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130892    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j2nd\" (UniqueName: \"kubernetes.io/projected/41115057-f034-42e6-96c0-1b398529a14c-kube-api-access-9j2nd\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.131027 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.130933    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41115057-f034-42e6-96c0-1b398529a14c-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.231913 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.231859    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41115057-f034-42e6-96c0-1b398529a14c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.231913 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.231920    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.232158 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.231948    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.232158 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.231964    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.232158 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.231981    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.232158 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.231999    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.232158 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232029    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.232158 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232082    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.232158 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232110    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.232158 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232135    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.232553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232165    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j2nd\" (UniqueName: \"kubernetes.io/projected/41115057-f034-42e6-96c0-1b398529a14c-kube-api-access-9j2nd\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.232553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232188    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41115057-f034-42e6-96c0-1b398529a14c-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.232553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232220    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.232553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232248    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5tzm\" (UniqueName: \"kubernetes.io/projected/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-kube-api-access-w5tzm\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.232553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232270    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.232553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232296    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.232553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232323    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.232553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232354    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41115057-f034-42e6-96c0-1b398529a14c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.232553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232430    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.232553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232449    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.232553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232452    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.233131 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232766    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.233131 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232732    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.233131 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.232849    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.233299 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.233198    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41115057-f034-42e6-96c0-1b398529a14c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.233385 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.233360    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.233469 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.233447    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.233520 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.233464    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.234552 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.234529    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.234863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.234845    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41115057-f034-42e6-96c0-1b398529a14c-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.235009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.234990    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41115057-f034-42e6-96c0-1b398529a14c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.235377 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.235361    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.255588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.255521    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41115057-f034-42e6-96c0-1b398529a14c-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.255588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.255577    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.255861 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.255835    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j2nd\" (UniqueName: \"kubernetes.io/projected/41115057-f034-42e6-96c0-1b398529a14c-kube-api-access-9j2nd\") pod \"maas-default-gateway-openshift-default-746ff8f49f-pb266\" (UID: \"41115057-f034-42e6-96c0-1b398529a14c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.255925 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.255860    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5tzm\" (UniqueName: \"kubernetes.io/projected/3eaa91d2-99cc-4e70-b98f-0c5a851dfb34-kube-api-access-w5tzm\") pod \"maas-default-gateway-openshift-default-746ff8f49f-lbpk9\" (UID: \"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.310456 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.310416    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:52.329996 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.329965    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:52.638069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.638044    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"]
Oct 29 23:26:52.640100 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:26:52.640070    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41115057_f034_42e6_96c0_1b398529a14c.slice/crio-1738d70183d4bad8e4e80c83183b6cd97a3f8b81303b6089b9ad985e2c36c097 WatchSource:0}: Error finding container 1738d70183d4bad8e4e80c83183b6cd97a3f8b81303b6089b9ad985e2c36c097: Status 404 returned error can't find the container with id 1738d70183d4bad8e4e80c83183b6cd97a3f8b81303b6089b9ad985e2c36c097
Oct 29 23:26:52.642234 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.642200    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:26:52.642322 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.642266    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:26:52.642322 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.642296    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:26:52.698137 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.698110    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"]
Oct 29 23:26:52.699022 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:26:52.698999    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eaa91d2_99cc_4e70_b98f_0c5a851dfb34.slice/crio-7baeb1048d1f3c662e0e22915be7afdd9286ef9879039f90d15a03be890228ac WatchSource:0}: Error finding container 7baeb1048d1f3c662e0e22915be7afdd9286ef9879039f90d15a03be890228ac: Status 404 returned error can't find the container with id 7baeb1048d1f3c662e0e22915be7afdd9286ef9879039f90d15a03be890228ac
Oct 29 23:26:52.701303 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.701270    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:26:52.701376 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.701349    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:26:52.701413 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.701391    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:26:52.843566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.843478    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" event={"ID":"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34","Type":"ContainerStarted","Data":"ce740f1ed79c9d9dfc83b253317a9efd342d0a1fe4bca0ca490b1f819411f7a6"}
Oct 29 23:26:52.843566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.843540    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" event={"ID":"3eaa91d2-99cc-4e70-b98f-0c5a851dfb34","Type":"ContainerStarted","Data":"7baeb1048d1f3c662e0e22915be7afdd9286ef9879039f90d15a03be890228ac"}
Oct 29 23:26:52.846164 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.846136    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266" event={"ID":"41115057-f034-42e6-96c0-1b398529a14c","Type":"ContainerStarted","Data":"5571e5eba27be6425b7a3d2f777edb9c8f8b2714c63f50b0a5368f6431277b8c"}
Oct 29 23:26:52.846278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.846189    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266" event={"ID":"41115057-f034-42e6-96c0-1b398529a14c","Type":"ContainerStarted","Data":"1738d70183d4bad8e4e80c83183b6cd97a3f8b81303b6089b9ad985e2c36c097"}
Oct 29 23:26:52.896814 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.896555    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" podStartSLOduration=1.896536177 podStartE2EDuration="1.896536177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:52.895526627 +0000 UTC m=+792.590635506" watchObservedRunningTime="2026-10-29 23:26:52.896536177 +0000 UTC m=+792.591645057"
Oct 29 23:26:52.957285 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:52.957206    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266" podStartSLOduration=1.9571906650000002 podStartE2EDuration="1.957190665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:52.95510948 +0000 UTC m=+792.650218373" watchObservedRunningTime="2026-10-29 23:26:52.957190665 +0000 UTC m=+792.652299543"
Oct 29 23:26:53.314818 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:53.311145    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:26:53.315305 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:53.314857    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-pb266 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:26:53.315305 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:53.314926    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266" podUID="41115057-f034-42e6-96c0-1b398529a14c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 29 23:26:53.334994 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:53.334395    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:26:53.334994 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:53.334532    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-lbpk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:26:53.334994 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:53.334573    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" podUID="3eaa91d2-99cc-4e70-b98f-0c5a851dfb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 29 23:26:54.314341 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:54.314302    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-pb266 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:26:54.314635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:54.314592    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266" podUID="41115057-f034-42e6-96c0-1b398529a14c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 29 23:26:54.334321 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:54.334271    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-lbpk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:26:54.335038 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:54.335004    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" podUID="3eaa91d2-99cc-4e70-b98f-0c5a851dfb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 29 23:26:55.318762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:55.315384    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-pb266 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:26:55.318762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:55.315473    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266" podUID="41115057-f034-42e6-96c0-1b398529a14c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 29 23:26:55.331307 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:55.331171    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-lbpk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:26:55.331307 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:55.331236    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" podUID="3eaa91d2-99cc-4e70-b98f-0c5a851dfb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 29 23:26:56.314762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:56.314177    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-pb266 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:26:56.314762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:56.314242    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266" podUID="41115057-f034-42e6-96c0-1b398529a14c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 29 23:26:56.335763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:56.332761    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-lbpk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:26:56.335763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:56.332825    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" podUID="3eaa91d2-99cc-4e70-b98f-0c5a851dfb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 29 23:26:57.324764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:57.320979    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-pb266 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:26:57.324764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:57.321038    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266" podUID="41115057-f034-42e6-96c0-1b398529a14c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 29 23:26:57.347762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:57.338114    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-lbpk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:26:57.347762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:57.338173    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" podUID="3eaa91d2-99cc-4e70-b98f-0c5a851dfb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 29 23:26:58.317761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:58.312165    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-pb266 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:26:58.317761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:58.312232    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266" podUID="41115057-f034-42e6-96c0-1b398529a14c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 29 23:26:58.338636 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:58.338598    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-lbpk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:26:58.339448 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:58.339325    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" podUID="3eaa91d2-99cc-4e70-b98f-0c5a851dfb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 29 23:26:59.311783 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:59.311726    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-pb266 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:26:59.312003 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:59.311800    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266" podUID="41115057-f034-42e6-96c0-1b398529a14c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 29 23:26:59.330591 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:59.330542    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-lbpk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:26:59.330799 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:26:59.330609    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" podUID="3eaa91d2-99cc-4e70-b98f-0c5a851dfb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 29 23:26:59.780373 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:26:59.780324    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:27:01.096407 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:01.096372    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:27:01.096968 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:01.096799    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:27:01.097855 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:01.097830    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9"
Oct 29 23:27:01.117093 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:01.117068    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:27:01.117420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:01.117402    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:27:01.118302 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:01.118281    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266"
Oct 29 23:27:29.779423 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:27:29.779385    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:27:34.132327 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.132274    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"]
Oct 29 23:27:34.135881 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.135861    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:27:34.141384 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.141356    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h6x5\" (UniqueName: \"kubernetes.io/projected/a539d29d-cb81-47d7-863d-e59027a12b58-kube-api-access-2h6x5\") pod \"payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg\" (UID: \"a539d29d-cb81-47d7-863d-e59027a12b58\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:27:34.141525 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.141430    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a539d29d-cb81-47d7-863d-e59027a12b58-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg\" (UID: \"a539d29d-cb81-47d7-863d-e59027a12b58\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:27:34.150183 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.150157    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-2a53c9\""
Oct 29 23:27:34.164021 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.163987    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"]
Oct 29 23:27:34.242586 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.242540    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h6x5\" (UniqueName: \"kubernetes.io/projected/a539d29d-cb81-47d7-863d-e59027a12b58-kube-api-access-2h6x5\") pod \"payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg\" (UID: \"a539d29d-cb81-47d7-863d-e59027a12b58\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:27:34.242776 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.242646    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a539d29d-cb81-47d7-863d-e59027a12b58-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg\" (UID: \"a539d29d-cb81-47d7-863d-e59027a12b58\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:27:34.243402 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.243379    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a539d29d-cb81-47d7-863d-e59027a12b58-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg\" (UID: \"a539d29d-cb81-47d7-863d-e59027a12b58\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:27:34.291557 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.291525    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h6x5\" (UniqueName: \"kubernetes.io/projected/a539d29d-cb81-47d7-863d-e59027a12b58-kube-api-access-2h6x5\") pod \"payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg\" (UID: \"a539d29d-cb81-47d7-863d-e59027a12b58\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:27:34.417393 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.417300    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:27:34.448790 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.448753    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:27:34.746544 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:34.746518    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"]
Oct 29 23:27:34.748452 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:27:34.748426    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda539d29d_cb81_47d7_863d_e59027a12b58.slice/crio-458962e60a173f3a23dc1cd2379ac58a852d6251b1084028075019fff2a1cca3 WatchSource:0}: Error finding container 458962e60a173f3a23dc1cd2379ac58a852d6251b1084028075019fff2a1cca3: Status 404 returned error can't find the container with id 458962e60a173f3a23dc1cd2379ac58a852d6251b1084028075019fff2a1cca3
Oct 29 23:27:35.062553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:35.062445    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg" event={"ID":"a539d29d-cb81-47d7-863d-e59027a12b58","Type":"ContainerStarted","Data":"458962e60a173f3a23dc1cd2379ac58a852d6251b1084028075019fff2a1cca3"}
Oct 29 23:27:37.075212 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.075125    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg" event={"ID":"a539d29d-cb81-47d7-863d-e59027a12b58","Type":"ContainerStarted","Data":"e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d"}
Oct 29 23:27:37.075601 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.075262    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:27:37.117358 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.117306    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg" podStartSLOduration=1.089976924 podStartE2EDuration="3.11729097s" totalImagesPullingTime="2.027314046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:37.115345597 +0000 UTC m=+836.810454477" watchObservedRunningTime="2026-10-29 23:27:37.11729097 +0000 UTC m=+836.812399859"
Oct 29 23:27:37.470355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.470317    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"]
Oct 29 23:27:37.474488 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.474465    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:27:37.493142 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.493109    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"]
Oct 29 23:27:37.503682 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.503650    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-ae1784\""
Oct 29 23:27:37.541405 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.541376    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-ae1784-dockercfg-hgdn9\""
Oct 29 23:27:37.573444 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.573406    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh\" (UID: \"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:27:37.573606 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.573465    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwgdq\" (UniqueName: \"kubernetes.io/projected/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-kube-api-access-pwgdq\") pod \"payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh\" (UID: \"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:27:37.674337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.674302    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh\" (UID: \"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:27:37.674568 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.674374    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwgdq\" (UniqueName: \"kubernetes.io/projected/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-kube-api-access-pwgdq\") pod \"payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh\" (UID: \"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:27:37.675348 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.675322    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh\" (UID: \"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:27:37.703480 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.703446    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwgdq\" (UniqueName: \"kubernetes.io/projected/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-kube-api-access-pwgdq\") pod \"payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh\" (UID: \"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:27:37.787184 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:37.787074    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:27:38.092640 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:38.092612    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"]
Oct 29 23:27:38.094676 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:27:38.094642    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bbff9d3_81f5_4f3f_9be1_afba3a8058ef.slice/crio-f56b5001ec7ec7cff66e2c5ecb8c79857679b134767e97ac7e71a2c60ae2d8b3 WatchSource:0}: Error finding container f56b5001ec7ec7cff66e2c5ecb8c79857679b134767e97ac7e71a2c60ae2d8b3: Status 404 returned error can't find the container with id f56b5001ec7ec7cff66e2c5ecb8c79857679b134767e97ac7e71a2c60ae2d8b3
Oct 29 23:27:39.088337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:39.088303    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh" event={"ID":"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef","Type":"ContainerStarted","Data":"e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377"}
Oct 29 23:27:39.088517 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:39.088357    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh" event={"ID":"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef","Type":"ContainerStarted","Data":"f56b5001ec7ec7cff66e2c5ecb8c79857679b134767e97ac7e71a2c60ae2d8b3"}
Oct 29 23:27:39.088517 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:39.088392    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:27:39.131168 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:39.131095    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh" podStartSLOduration=2.131077033 podStartE2EDuration="2.131077033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:39.128875974 +0000 UTC m=+838.823984879" watchObservedRunningTime="2026-10-29 23:27:39.131077033 +0000 UTC m=+838.826185933"
Oct 29 23:27:39.678861 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:39.678828    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:27:44.362425 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.362390    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"]
Oct 29 23:27:44.367134 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.367115    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.396180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.396150    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-ae1364-serving-cert\""
Oct 29 23:27:44.396319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.396208    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 29 23:27:44.396319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.396269    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-k7tlz\""
Oct 29 23:27:44.399540 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.399518    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"]
Oct 29 23:27:44.439781 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.439730    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m687\" (UniqueName: \"kubernetes.io/projected/df0c1ec4-67dd-424e-97bf-91e504744d37-kube-api-access-5m687\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.439942 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.439837    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df0c1ec4-67dd-424e-97bf-91e504744d37-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.439942 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.439877    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-metrics-certs\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.440029 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.439957    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.457311 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.457285    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:27:44.457453 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.457315    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:27:44.541330 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.541291    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m687\" (UniqueName: \"kubernetes.io/projected/df0c1ec4-67dd-424e-97bf-91e504744d37-kube-api-access-5m687\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.541508 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.541349    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df0c1ec4-67dd-424e-97bf-91e504744d37-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.541508 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.541382    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-metrics-certs\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.541508 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.541425    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.542050 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.542026    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df0c1ec4-67dd-424e-97bf-91e504744d37-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.543860 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.543837    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-metrics-certs\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.543949 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.543912    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.567523 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.567497    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m687\" (UniqueName: \"kubernetes.io/projected/df0c1ec4-67dd-424e-97bf-91e504744d37-kube-api-access-5m687\") pod \"maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.680223 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.680186    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:44.986371 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:44.986342    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"]
Oct 29 23:27:44.988940 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:27:44.988904    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0c1ec4_67dd_424e_97bf_91e504744d37.slice/crio-d606f844fb097c7f8acf047ff2ffba2134eaa05aedfb96aae83748ac24835114 WatchSource:0}: Error finding container d606f844fb097c7f8acf047ff2ffba2134eaa05aedfb96aae83748ac24835114: Status 404 returned error can't find the container with id d606f844fb097c7f8acf047ff2ffba2134eaa05aedfb96aae83748ac24835114
Oct 29 23:27:45.119640 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:45.119606    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr" event={"ID":"df0c1ec4-67dd-424e-97bf-91e504744d37","Type":"ContainerStarted","Data":"d606f844fb097c7f8acf047ff2ffba2134eaa05aedfb96aae83748ac24835114"}
Oct 29 23:27:45.490378 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:45.490336    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:27:48.083901 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:48.083874    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:27:48.140166 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:48.140131    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr" event={"ID":"df0c1ec4-67dd-424e-97bf-91e504744d37","Type":"ContainerStarted","Data":"36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2"}
Oct 29 23:27:48.140366 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:48.140276    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:48.191902 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:48.191845    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr" podStartSLOduration=1.82418912 podStartE2EDuration="4.191825199s" totalImagesPullingTime="2.367636079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:48.189317298 +0000 UTC m=+847.884426179" watchObservedRunningTime="2026-10-29 23:27:48.191825199 +0000 UTC m=+847.886934079"
Oct 29 23:27:49.221048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.221011    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"]
Oct 29 23:27:49.225266 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.225242    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.244351 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.244323    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"]
Oct 29 23:27:49.253725 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.253697    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-53c44e\"/\"kube-root-ca.crt\""
Oct 29 23:27:49.253725 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.253697    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-53c44e\"/\"openshift-service-ca.crt\""
Oct 29 23:27:49.255897 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.255869    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-53c44e\"/\"default-dockercfg-z6kgj\""
Oct 29 23:27:49.256009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.255951    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-53c44e\"/\"simulator-w5-53c44e-kserve-self-signed-certs\""
Oct 29 23:27:49.292518 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.292483    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-dshm\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.292710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.292526    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-model-cache\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.292710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.292545    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-home\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.292710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.292619    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6465\" (UniqueName: \"kubernetes.io/projected/2a3d9093-7037-44e8-a741-743b6b1d4665-kube-api-access-q6465\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.292710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.292700    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a3d9093-7037-44e8-a741-743b6b1d4665-tls-certs\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.292910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.292765    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-tmp-dir\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.393626 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.393590    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-dshm\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.393626 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.393633    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-model-cache\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.393894 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.393652    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-home\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.393894 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.393799    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6465\" (UniqueName: \"kubernetes.io/projected/2a3d9093-7037-44e8-a741-743b6b1d4665-kube-api-access-q6465\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.393894 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.393842    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a3d9093-7037-44e8-a741-743b6b1d4665-tls-certs\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.393894 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.393876    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-tmp-dir\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.394096 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.394028    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-model-cache\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.394152 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.394109    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-home\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.394217 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.394196    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-tmp-dir\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.396021 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.395992    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-dshm\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.396227 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.396208    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a3d9093-7037-44e8-a741-743b6b1d4665-tls-certs\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.417139 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.417115    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6465\" (UniqueName: \"kubernetes.io/projected/2a3d9093-7037-44e8-a741-743b6b1d4665-kube-api-access-q6465\") pod \"simulator-w5-53c44e-kserve-697b6d6464-tp7dh\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") " pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.539718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.539638    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:49.837813 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:49.837788    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"]
Oct 29 23:27:49.839850 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:27:49.839823    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a3d9093_7037_44e8_a741_743b6b1d4665.slice/crio-68fab848718f28828d804e470c983df1f246cab39d05a7ed93929f118d5d0138 WatchSource:0}: Error finding container 68fab848718f28828d804e470c983df1f246cab39d05a7ed93929f118d5d0138: Status 404 returned error can't find the container with id 68fab848718f28828d804e470c983df1f246cab39d05a7ed93929f118d5d0138
Oct 29 23:27:50.097500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:50.097419    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:27:50.155660 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:50.155606    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" event={"ID":"2a3d9093-7037-44e8-a741-743b6b1d4665","Type":"ContainerStarted","Data":"68fab848718f28828d804e470c983df1f246cab39d05a7ed93929f118d5d0138"}
Oct 29 23:27:51.165173 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:51.165137    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" event={"ID":"2a3d9093-7037-44e8-a741-743b6b1d4665","Type":"ContainerStarted","Data":"e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7"}
Oct 29 23:27:51.217018 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:51.216917    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podStartSLOduration=1.105712122 podStartE2EDuration="2.216901718s" totalImagesPullingTime="1.111189596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:51.214902101 +0000 UTC m=+850.910010980" watchObservedRunningTime="2026-10-29 23:27:51.216901718 +0000 UTC m=+850.912010596"
Oct 29 23:27:51.540119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:51.540029    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:53.154471 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:53.154436    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:27:54.150908 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:54.150871    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:27:59.540205 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:59.540170    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:59.552853 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:27:59.552826    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:27:59.779822 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:27:59.779789    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:28:00.224229 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.224199    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:28:00.787624 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.787591    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"]
Oct 29 23:28:00.791441 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.791415    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:00.821688 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.821656    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-92f6b3\"/\"openshift-service-ca.crt\""
Oct 29 23:28:00.821864 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.821760    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-92f6b3\"/\"kube-root-ca.crt\""
Oct 29 23:28:00.821864 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.821762    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-92f6b3\"/\"default-dockercfg-dtsbz\""
Oct 29 23:28:00.822127 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.822108    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-92f6b3\"/\"simulator-w3-92f6b3-kserve-self-signed-certs\""
Oct 29 23:28:00.830408 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.830381    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"]
Oct 29 23:28:00.904002 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.903967    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-dshm\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:00.904002 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.904004    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9pqx\" (UniqueName: \"kubernetes.io/projected/6fd45077-7c35-4519-8a8f-554c5fbf4128-kube-api-access-q9pqx\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:00.904267 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.904049    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-tmp-dir\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:00.904267 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.904179    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-model-cache\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:00.904267 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.904220    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fd45077-7c35-4519-8a8f-554c5fbf4128-tls-certs\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:00.904267 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:00.904243    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-home\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.005064 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.005030    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-dshm\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.005064 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.005066    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9pqx\" (UniqueName: \"kubernetes.io/projected/6fd45077-7c35-4519-8a8f-554c5fbf4128-kube-api-access-q9pqx\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.005321 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.005107    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-tmp-dir\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.005321 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.005134    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-model-cache\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.005321 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.005159    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fd45077-7c35-4519-8a8f-554c5fbf4128-tls-certs\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.005321 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.005192    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-home\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.005577 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.005552    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-tmp-dir\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.005644 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.005572    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-model-cache\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.005644 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.005605    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-home\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.007349 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.007330    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-dshm\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.007619 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.007600    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fd45077-7c35-4519-8a8f-554c5fbf4128-tls-certs\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.030009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.029975    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9pqx\" (UniqueName: \"kubernetes.io/projected/6fd45077-7c35-4519-8a8f-554c5fbf4128-kube-api-access-q9pqx\") pod \"simulator-w3-92f6b3-kserve-84dd898bb6-54fqp\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") " pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.103504 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.103421    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:01.412218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:01.412189    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"]
Oct 29 23:28:01.413804 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:28:01.413773    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fd45077_7c35_4519_8a8f_554c5fbf4128.slice/crio-f481fcbf37e59d200dafbcd5f91b088d0e1eca79638a653a044b06cd4b7edd5d WatchSource:0}: Error finding container f481fcbf37e59d200dafbcd5f91b088d0e1eca79638a653a044b06cd4b7edd5d: Status 404 returned error can't find the container with id f481fcbf37e59d200dafbcd5f91b088d0e1eca79638a653a044b06cd4b7edd5d
Oct 29 23:28:02.225461 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:02.225426    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" event={"ID":"6fd45077-7c35-4519-8a8f-554c5fbf4128","Type":"ContainerStarted","Data":"da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24"}
Oct 29 23:28:02.225461 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:02.225466    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" event={"ID":"6fd45077-7c35-4519-8a8f-554c5fbf4128","Type":"ContainerStarted","Data":"f481fcbf37e59d200dafbcd5f91b088d0e1eca79638a653a044b06cd4b7edd5d"}
Oct 29 23:28:02.275070 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:02.275012    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" podStartSLOduration=2.274997069 podStartE2EDuration="2.274997069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:02.273415333 +0000 UTC m=+861.968524213" watchObservedRunningTime="2026-10-29 23:28:02.274997069 +0000 UTC m=+861.970106001"
Oct 29 23:28:03.103587 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:03.103545    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:05.939090 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:05.939053    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:28:11.104084 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:11.104045    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:11.116612 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:11.116586    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:11.281857 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:11.281831    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:28:14.718866 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.718831    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"]
Oct 29 23:28:14.726397 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.726376    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.745482 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.745452    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"]
Oct 29 23:28:14.755246 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.755210    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-aa7e96\"/\"kube-root-ca.crt\""
Oct 29 23:28:14.755246 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.755213    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-aa7e96\"/\"openshift-service-ca.crt\""
Oct 29 23:28:14.762426 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.762401    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-aa7e96\"/\"default-dockercfg-xr4t2\""
Oct 29 23:28:14.762426 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.762417    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-aa7e96\"/\"simulator-w4-aa7e96-kserve-self-signed-certs\""
Oct 29 23:28:14.833654 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.833620    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-home\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.833861 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.833660    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tmp-dir\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.833861 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.833686    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tls-certs\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.833941 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.833893    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d4z6\" (UniqueName: \"kubernetes.io/projected/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-kube-api-access-4d4z6\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.833976 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.833956    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-model-cache\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.834048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.834023    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-dshm\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.934850 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.934815    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-home\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.934850 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.934860    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tmp-dir\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.935182 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.934878    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tls-certs\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.935182 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.934936    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d4z6\" (UniqueName: \"kubernetes.io/projected/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-kube-api-access-4d4z6\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.935182 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.934986    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-model-cache\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.935182 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.935019    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-dshm\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.935409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.935363    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-home\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.935409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.935386    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tmp-dir\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.935507 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.935407    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-model-cache\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.937383 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.937358    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-dshm\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.937628 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.937610    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tls-certs\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:14.967177 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:14.967148    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d4z6\" (UniqueName: \"kubernetes.io/projected/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-kube-api-access-4d4z6\") pod \"simulator-w4-aa7e96-kserve-69c84f6756-75hxv\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") " pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:15.039896 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:15.039804    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:15.343006 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:15.342977    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"]
Oct 29 23:28:15.345273 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:28:15.345238    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0d3ee3e_07bd_4868_a39a_dccab1fc6222.slice/crio-1655f6e66501a3d4190592eb844069e9b3107535c6029bbcca04eebbe2d9293f WatchSource:0}: Error finding container 1655f6e66501a3d4190592eb844069e9b3107535c6029bbcca04eebbe2d9293f: Status 404 returned error can't find the container with id 1655f6e66501a3d4190592eb844069e9b3107535c6029bbcca04eebbe2d9293f
Oct 29 23:28:16.306493 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:16.306451    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" event={"ID":"a0d3ee3e-07bd-4868-a39a-dccab1fc6222","Type":"ContainerStarted","Data":"cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4"}
Oct 29 23:28:16.306493 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:16.306499    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" event={"ID":"a0d3ee3e-07bd-4868-a39a-dccab1fc6222","Type":"ContainerStarted","Data":"1655f6e66501a3d4190592eb844069e9b3107535c6029bbcca04eebbe2d9293f"}
Oct 29 23:28:16.355945 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:16.355894    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podStartSLOduration=2.355878144 podStartE2EDuration="2.355878144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:16.354129315 +0000 UTC m=+876.049238209" watchObservedRunningTime="2026-10-29 23:28:16.355878144 +0000 UTC m=+876.050987023"
Oct 29 23:28:17.040725 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:17.040684    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:18.658499 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.658453    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"]
Oct 29 23:28:18.663213 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.663186    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.672381 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.672354    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f002fa2-9040-468f-9042-6d2c8abec047-tls-certs\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.672515 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.672399    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-tmp-dir\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.672515 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.672417    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-home\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.672624 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.672578    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-model-cache\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.672720 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.672699    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmk78\" (UniqueName: \"kubernetes.io/projected/6f002fa2-9040-468f-9042-6d2c8abec047-kube-api-access-nmk78\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.672798 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.672753    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-dshm\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.687217 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.687186    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-2a53c9\"/\"simulator-w1-2a53c9-kserve-self-signed-certs\""
Oct 29 23:28:18.699252 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.699220    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-2a53c9\"/\"default-dockercfg-8zffd\""
Oct 29 23:28:18.701025 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.700995    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-2a53c9\"/\"openshift-service-ca.crt\""
Oct 29 23:28:18.701184 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.701068    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-2a53c9\"/\"kube-root-ca.crt\""
Oct 29 23:28:18.702598 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.702573    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"]
Oct 29 23:28:18.773655 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.773609    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmk78\" (UniqueName: \"kubernetes.io/projected/6f002fa2-9040-468f-9042-6d2c8abec047-kube-api-access-nmk78\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.773898 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.773665    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-dshm\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.773898 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.773714    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f002fa2-9040-468f-9042-6d2c8abec047-tls-certs\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.773898 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.773789    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-tmp-dir\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.773898 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.773817    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-home\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.773898 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.773868    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-model-cache\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.774260 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.774235    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-tmp-dir\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.774325 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.774247    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-home\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.774325 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.774304    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-model-cache\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.776098 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.776073    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-dshm\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.776377 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.776362    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f002fa2-9040-468f-9042-6d2c8abec047-tls-certs\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.805197 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.805163    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmk78\" (UniqueName: \"kubernetes.io/projected/6f002fa2-9040-468f-9042-6d2c8abec047-kube-api-access-nmk78\") pod \"simulator-w1-2a53c9-kserve-749cd577bc-snldk\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") " pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:18.978514 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:18.978412    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:19.289675 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:19.289622    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"]
Oct 29 23:28:19.292371 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:28:19.292342    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f002fa2_9040_468f_9042_6d2c8abec047.slice/crio-e84d68c8180623e76bd0ba7484f4bb5b86e7bc9c2f000021481849c885b8c050 WatchSource:0}: Error finding container e84d68c8180623e76bd0ba7484f4bb5b86e7bc9c2f000021481849c885b8c050: Status 404 returned error can't find the container with id e84d68c8180623e76bd0ba7484f4bb5b86e7bc9c2f000021481849c885b8c050
Oct 29 23:28:19.325167 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:19.325135    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" event={"ID":"6f002fa2-9040-468f-9042-6d2c8abec047","Type":"ContainerStarted","Data":"e84d68c8180623e76bd0ba7484f4bb5b86e7bc9c2f000021481849c885b8c050"}
Oct 29 23:28:19.888437 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:19.888397    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:28:20.331760 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:20.331656    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" event={"ID":"6f002fa2-9040-468f-9042-6d2c8abec047","Type":"ContainerStarted","Data":"c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1"}
Oct 29 23:28:20.382509 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:20.382460    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podStartSLOduration=2.382443335 podStartE2EDuration="2.382443335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:20.379019877 +0000 UTC m=+880.074128756" watchObservedRunningTime="2026-10-29 23:28:20.382443335 +0000 UTC m=+880.077552214"
Oct 29 23:28:20.978788 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:20.978725    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:25.040659 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:25.040624    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:25.053284 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:25.053258    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:25.370821 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:25.370726    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:28:28.979190 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:28.979149    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:28.998419 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:28.998389    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:29.398436 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:29.398402    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:28:29.779298 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:28:29.779210    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:28:33.970010 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:33.969973    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"]
Oct 29 23:28:33.973673 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:33.973653    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:33.990175 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:33.990148    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-ae1784\"/\"default-dockercfg-95pjs\""
Oct 29 23:28:34.002323 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.002292    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-ae1784\"/\"openshift-service-ca.crt\""
Oct 29 23:28:34.002519 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.002498    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-ae1784\"/\"simulator-w6-ae1784-kserve-self-signed-certs\""
Oct 29 23:28:34.002613 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.002597    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-ae1784\"/\"kube-root-ca.crt\""
Oct 29 23:28:34.004584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.004557    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"]
Oct 29 23:28:34.027822 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.027788    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-tmp-dir\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.027967 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.027827    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-dshm\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.027967 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.027901    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b29afd-299b-4dfc-b78f-73981c1c343a-tls-certs\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.027967 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.027950    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rh75\" (UniqueName: \"kubernetes.io/projected/d3b29afd-299b-4dfc-b78f-73981c1c343a-kube-api-access-6rh75\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.028093 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.028077    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-model-cache\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.028129 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.028118    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-home\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.129175 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.129141    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-home\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.129364 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.129196    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-tmp-dir\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.129364 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.129229    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-dshm\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.129364 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.129260    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b29afd-299b-4dfc-b78f-73981c1c343a-tls-certs\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.129364 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.129324    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rh75\" (UniqueName: \"kubernetes.io/projected/d3b29afd-299b-4dfc-b78f-73981c1c343a-kube-api-access-6rh75\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.129586 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.129415    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-model-cache\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.129643 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.129601    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-home\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.129789 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.129761    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-tmp-dir\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.129915 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.129860    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-model-cache\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.131773 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.131724    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-dshm\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.132223 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.132202    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b29afd-299b-4dfc-b78f-73981c1c343a-tls-certs\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.153341 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.153303    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rh75\" (UniqueName: \"kubernetes.io/projected/d3b29afd-299b-4dfc-b78f-73981c1c343a-kube-api-access-6rh75\") pod \"simulator-w6-ae1784-kserve-7db7fbc88c-9rwff\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") " pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.287961 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.287872    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:34.590927 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.590905    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"]
Oct 29 23:28:34.592674 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:28:34.592641    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3b29afd_299b_4dfc_b78f_73981c1c343a.slice/crio-9ddbcfeb41de56e90420c04de61976f546f0b8f76bf1d0f5bdebfee51d059a22 WatchSource:0}: Error finding container 9ddbcfeb41de56e90420c04de61976f546f0b8f76bf1d0f5bdebfee51d059a22: Status 404 returned error can't find the container with id 9ddbcfeb41de56e90420c04de61976f546f0b8f76bf1d0f5bdebfee51d059a22
Oct 29 23:28:34.712192 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:34.712159    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:28:35.416113 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:35.416066    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" event={"ID":"d3b29afd-299b-4dfc-b78f-73981c1c343a","Type":"ContainerStarted","Data":"348dbc4f806b3acb06e429977e5646b140b926736b523feded21c29078f52200"}
Oct 29 23:28:35.416604 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:35.416128    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" event={"ID":"d3b29afd-299b-4dfc-b78f-73981c1c343a","Type":"ContainerStarted","Data":"9ddbcfeb41de56e90420c04de61976f546f0b8f76bf1d0f5bdebfee51d059a22"}
Oct 29 23:28:35.470959 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:35.470854    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podStartSLOduration=2.470835078 podStartE2EDuration="2.470835078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:35.470563185 +0000 UTC m=+895.165672064" watchObservedRunningTime="2026-10-29 23:28:35.470835078 +0000 UTC m=+895.165943958"
Oct 29 23:28:36.288532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:36.288490    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:40.535798 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.535767    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"]
Oct 29 23:28:40.539923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.539898    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.575642 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.575604    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b978a0\"/\"default-dockercfg-zzj7v\""
Oct 29 23:28:40.575844 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.575690    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b978a0\"/\"premium-w0-b978a0-kserve-self-signed-certs\""
Oct 29 23:28:40.575917 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.575888    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b978a0\"/\"openshift-service-ca.crt\""
Oct 29 23:28:40.575973 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.575925    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b978a0\"/\"kube-root-ca.crt\""
Oct 29 23:28:40.578940 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.578915    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"]
Oct 29 23:28:40.697395 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.697354    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-home\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.697584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.697414    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tmp-dir\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.697584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.697560    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjbdm\" (UniqueName: \"kubernetes.io/projected/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-kube-api-access-cjbdm\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.697681 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.697601    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-dshm\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.697681 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.697666    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-model-cache\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.697785 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.697762    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tls-certs\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.799411 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.799306    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjbdm\" (UniqueName: \"kubernetes.io/projected/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-kube-api-access-cjbdm\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.799411 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.799365    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-dshm\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.799411 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.799411    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-model-cache\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.799706 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.799450    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tls-certs\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.799706 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.799494    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-home\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.799706 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.799529    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tmp-dir\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.800017 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.799988    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-home\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.800100 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.800013    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tmp-dir\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.800283 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.800253    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-model-cache\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.802045 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.802017    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-dshm\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.802221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.802202    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tls-certs\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.824168 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.824144    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjbdm\" (UniqueName: \"kubernetes.io/projected/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-kube-api-access-cjbdm\") pod \"premium-w0-b978a0-kserve-5c845494dc-pnfdd\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") " pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:40.854875 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:40.854847    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:41.163186 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:41.163040    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"]
Oct 29 23:28:41.165534 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:28:41.165504    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod658cbe09_1fd9_4438_a5ba_ef3d98bfe46d.slice/crio-9dd1d2a7cefac7f531eba7f0bb0295890c9f0bd9874db77ab50aa5c1fd2f6c0c WatchSource:0}: Error finding container 9dd1d2a7cefac7f531eba7f0bb0295890c9f0bd9874db77ab50aa5c1fd2f6c0c: Status 404 returned error can't find the container with id 9dd1d2a7cefac7f531eba7f0bb0295890c9f0bd9874db77ab50aa5c1fd2f6c0c
Oct 29 23:28:41.448632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:41.448545    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" event={"ID":"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d","Type":"ContainerStarted","Data":"d1c2cf97632ca058938093818af923436cad5188ce0d09b0bdccc2dbc911ad85"}
Oct 29 23:28:41.448632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:41.448590    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" event={"ID":"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d","Type":"ContainerStarted","Data":"9dd1d2a7cefac7f531eba7f0bb0295890c9f0bd9874db77ab50aa5c1fd2f6c0c"}
Oct 29 23:28:41.492928 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:41.492871    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podStartSLOduration=1.49285728 podStartE2EDuration="1.49285728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:41.49260616 +0000 UTC m=+901.187715050" watchObservedRunningTime="2026-10-29 23:28:41.49285728 +0000 UTC m=+901.187966158"
Oct 29 23:28:41.855507 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:41.855419    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:44.288368 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:44.288331    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:44.301839 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:44.301811    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:44.474173 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:44.474140    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:28:48.431154 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:48.431124    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:28:50.855195 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:50.855111    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:50.867367 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:50.867338    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:51.512524 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:51.512499    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:28:54.945086 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:54.945053    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"]
Oct 29 23:28:54.950328 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:54.950306    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:54.978544 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:54.978511    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-aa7e96\"/\"premium-w4-aa7e96-kserve-self-signed-certs\""
Oct 29 23:28:54.981136 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:54.981107    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"]
Oct 29 23:28:55.046071 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.046032    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a292574a-72a5-427c-a344-255fac8aa567-tls-certs\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.046246 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.046093    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-home\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.046246 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.046200    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmr2q\" (UniqueName: \"kubernetes.io/projected/a292574a-72a5-427c-a344-255fac8aa567-kube-api-access-nmr2q\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.046246 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.046239    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-dshm\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.046417 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.046391    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-tmp-dir\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.046480 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.046440    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-model-cache\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.147404 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.147358    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-tmp-dir\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.147602 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.147414    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-model-cache\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.147602 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.147481    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a292574a-72a5-427c-a344-255fac8aa567-tls-certs\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.147602 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.147522    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-home\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.147602 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.147586    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmr2q\" (UniqueName: \"kubernetes.io/projected/a292574a-72a5-427c-a344-255fac8aa567-kube-api-access-nmr2q\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.147861 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.147626    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-dshm\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.147937 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.147916    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-tmp-dir\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.147989 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.147958    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-model-cache\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.148173 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.148142    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-home\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.150110 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.150083    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-dshm\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.150343 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.150319    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a292574a-72a5-427c-a344-255fac8aa567-tls-certs\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.171312 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.171281    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmr2q\" (UniqueName: \"kubernetes.io/projected/a292574a-72a5-427c-a344-255fac8aa567-kube-api-access-nmr2q\") pod \"premium-w4-aa7e96-kserve-6985bb7df7-jbxjg\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") " pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.262696 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.262616    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:55.561937 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:55.561910    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"]
Oct 29 23:28:55.564537 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:28:55.564506    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda292574a_72a5_427c_a344_255fac8aa567.slice/crio-18e0190dd42548ebf16b88aab524a1f414c2540664bb7671c8cca1f6dc4539d7 WatchSource:0}: Error finding container 18e0190dd42548ebf16b88aab524a1f414c2540664bb7671c8cca1f6dc4539d7: Status 404 returned error can't find the container with id 18e0190dd42548ebf16b88aab524a1f414c2540664bb7671c8cca1f6dc4539d7
Oct 29 23:28:56.530617 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:56.530576    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" event={"ID":"a292574a-72a5-427c-a344-255fac8aa567","Type":"ContainerStarted","Data":"02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23"}
Oct 29 23:28:56.530617 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:56.530623    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" event={"ID":"a292574a-72a5-427c-a344-255fac8aa567","Type":"ContainerStarted","Data":"18e0190dd42548ebf16b88aab524a1f414c2540664bb7671c8cca1f6dc4539d7"}
Oct 29 23:28:56.624935 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:56.624884    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podStartSLOduration=2.624869558 podStartE2EDuration="2.624869558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:56.623954136 +0000 UTC m=+916.319063014" watchObservedRunningTime="2026-10-29 23:28:56.624869558 +0000 UTC m=+916.319978485"
Oct 29 23:28:57.263027 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:28:57.262981    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:28:59.779977 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:28:59.779938    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:29:05.205927 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.205893    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"]
Oct 29 23:29:05.209243 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.209221    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.226266 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.226241    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-ae1364\"/\"premium-w2-ae1364-kserve-self-signed-certs\""
Oct 29 23:29:05.238401 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.238347    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-ae1364\"/\"openshift-service-ca.crt\""
Oct 29 23:29:05.238401 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.238374    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-ae1364\"/\"default-dockercfg-kphlj\""
Oct 29 23:29:05.238631 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.238556    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-ae1364\"/\"kube-root-ca.crt\""
Oct 29 23:29:05.240708 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.240679    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"]
Oct 29 23:29:05.249854 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.249833    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tmp-dir\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.249976 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.249885    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-home\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.249976 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.249903    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-dshm\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.250049 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.250031    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tls-certs\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.250083 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.250056    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-model-cache\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.250161 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.250142    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr2nt\" (UniqueName: \"kubernetes.io/projected/a37b6f4c-bacc-4584-bbf2-b2414fa85897-kube-api-access-zr2nt\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.263283 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.263260    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:29:05.276202 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.276174    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:29:05.351372 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.351331    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-home\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.351372 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.351372    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-dshm\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.351623 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.351406    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tls-certs\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.351623 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.351451    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-model-cache\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.351623 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.351530    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr2nt\" (UniqueName: \"kubernetes.io/projected/a37b6f4c-bacc-4584-bbf2-b2414fa85897-kube-api-access-zr2nt\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.351813 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.351658    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tmp-dir\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.351875 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.351815    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-home\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.352037 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.352016    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tmp-dir\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.352271 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.352251    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-model-cache\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.354081 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.354052    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tls-certs\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.354191 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.354107    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-dshm\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.379685 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.379650    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr2nt\" (UniqueName: \"kubernetes.io/projected/a37b6f4c-bacc-4584-bbf2-b2414fa85897-kube-api-access-zr2nt\") pod \"premium-w2-ae1364-kserve-74f4d67488-gws94\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") " pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.523765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.523660    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:05.596340 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.596307    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:29:05.828092 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:05.828061    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"]
Oct 29 23:29:05.829282 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:29:05.829251    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda37b6f4c_bacc_4584_bbf2_b2414fa85897.slice/crio-145a925a247962f4a1419435399dc30a197703c2ecadb4f1216a41434e76fa6b WatchSource:0}: Error finding container 145a925a247962f4a1419435399dc30a197703c2ecadb4f1216a41434e76fa6b: Status 404 returned error can't find the container with id 145a925a247962f4a1419435399dc30a197703c2ecadb4f1216a41434e76fa6b
Oct 29 23:29:06.586303 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:06.586271    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" event={"ID":"a37b6f4c-bacc-4584-bbf2-b2414fa85897","Type":"ContainerStarted","Data":"538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440"}
Oct 29 23:29:06.586303 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:06.586309    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" event={"ID":"a37b6f4c-bacc-4584-bbf2-b2414fa85897","Type":"ContainerStarted","Data":"145a925a247962f4a1419435399dc30a197703c2ecadb4f1216a41434e76fa6b"}
Oct 29 23:29:06.631990 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:06.631911    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podStartSLOduration=1.63189256 podStartE2EDuration="1.63189256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:06.62950882 +0000 UTC m=+926.324617697" watchObservedRunningTime="2026-10-29 23:29:06.63189256 +0000 UTC m=+926.327001445"
Oct 29 23:29:07.524786 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:07.524717    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:07.674593 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:07.674550    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:29:15.524064 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:15.524026    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:15.536644 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:15.536614    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:15.652614 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:15.652586    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:29:23.731505 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.731471    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"]
Oct 29 23:29:23.736349 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.736328    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.764394 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.764356    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-53c44e-openshift-default-dockercfg-hgv77\""
Oct 29 23:29:23.773102 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.773046    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"]
Oct 29 23:29:23.844035 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.843996    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/696b829d-a17c-48d8-9cbc-ec567e7add16-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.844221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.844041    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.844221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.844198    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.844299 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.844222    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.844299 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.844251    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.844299 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.844292    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.844395 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.844311    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.844395 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.844369    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7tjw\" (UniqueName: \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-kube-api-access-g7tjw\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.844455 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.844393    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.945268 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945225    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.945268 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945264    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.945560 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945288    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.945560 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945306    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.945560 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945332    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.945560 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945371    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7tjw\" (UniqueName: \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-kube-api-access-g7tjw\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.945560 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945410    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.945875 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945707    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/696b829d-a17c-48d8-9cbc-ec567e7add16-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.945875 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945793    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.945875 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945839    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.945875 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945854    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.946091 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.945917    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.946205 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.946181    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.946445 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.946411    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/696b829d-a17c-48d8-9cbc-ec567e7add16-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.947907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.947880    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:23.948049 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:23.948030    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:24.009380 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:24.009290    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:24.018340 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:24.018309    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7tjw\" (UniqueName: \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-kube-api-access-g7tjw\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:24.050215 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:24.050179    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:24.393865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:24.393812    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"]
Oct 29 23:29:24.396347 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:29:24.396313    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod696b829d_a17c_48d8_9cbc_ec567e7add16.slice/crio-14909d360525f159ec5d973164552caa31a852e1cde5c9c5ac2e3faf64e0853a WatchSource:0}: Error finding container 14909d360525f159ec5d973164552caa31a852e1cde5c9c5ac2e3faf64e0853a: Status 404 returned error can't find the container with id 14909d360525f159ec5d973164552caa31a852e1cde5c9c5ac2e3faf64e0853a
Oct 29 23:29:24.398600 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:24.398570    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:29:24.398664 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:24.398635    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:29:24.398792 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:24.398668    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:29:24.688718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:24.688620    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" event={"ID":"696b829d-a17c-48d8-9cbc-ec567e7add16","Type":"ContainerStarted","Data":"40455e2620c31d6b8511e2975c950967076425c74b309fdde4e3a52b5ee49196"}
Oct 29 23:29:24.688718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:24.688691    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" event={"ID":"696b829d-a17c-48d8-9cbc-ec567e7add16","Type":"ContainerStarted","Data":"14909d360525f159ec5d973164552caa31a852e1cde5c9c5ac2e3faf64e0853a"}
Oct 29 23:29:24.737263 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:24.737204    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" podStartSLOduration=1.73718726 podStartE2EDuration="1.73718726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:24.734786194 +0000 UTC m=+944.429895073" watchObservedRunningTime="2026-10-29 23:29:24.73718726 +0000 UTC m=+944.432296138"
Oct 29 23:29:25.051292 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:25.051200    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:25.052951 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:25.052919    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 29 23:29:25.053095 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:25.052968    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 29 23:29:26.051293 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:26.051248    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 29 23:29:26.051702 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:26.051318    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 29 23:29:27.050771 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:27.050725    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 29 23:29:27.050954 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:27.050794    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 29 23:29:28.050811 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:28.050772    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 29 23:29:28.051240 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:28.050832    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 29 23:29:29.051080 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:29.051045    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 29 23:29:29.051446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:29.051102    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 29 23:29:29.780197 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:29:29.780160    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:29:30.051648 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:30.051556    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 29 23:29:30.051648 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:30.051611    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 29 23:29:31.075944 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:31.075911    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:31.076780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:31.076732    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:31.078772 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:31.078751    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:29:31.589421 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:31.589387    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:29:39.385190 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.385153    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"]
Oct 29 23:29:39.389066 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.389043    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.412081 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.412050    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-ae1364-openshift-default-dockercfg-9nkvr\""
Oct 29 23:29:39.420936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.420903    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"]
Oct 29 23:29:39.440827 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.440794    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49d459ae-07ae-4c53-aae6-934c25119c6c-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.441003 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.440880    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.441003 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.440934    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.441125 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.440999    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.441125 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.441022    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.441125 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.441077    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.441125 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.441114    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.441326 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.441149    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.441326 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.441198    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k774h\" (UniqueName: \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-kube-api-access-k774h\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.542610 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.542579    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.542814 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.542627    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.542814 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.542688    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.542814 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.542711    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.542814 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.542781    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.543051 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.542815    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.543051 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.542846    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.543051 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.542890    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k774h\" (UniqueName: \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-kube-api-access-k774h\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.543051 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.542935    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49d459ae-07ae-4c53-aae6-934c25119c6c-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.543277 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.543229    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.543277 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.543255    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.543561 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.543538    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.543631 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.543580    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.543631 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.543598    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49d459ae-07ae-4c53-aae6-934c25119c6c-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.545307 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.545277    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.545554 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.545532    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.565923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.565895    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.566054 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.565975    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k774h\" (UniqueName: \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-kube-api-access-k774h\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:39.702590 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:39.702493    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:40.016730 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:40.016698    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"]
Oct 29 23:29:40.018774 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:29:40.018731    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d459ae_07ae_4c53_aae6_934c25119c6c.slice/crio-ad64feb6b1a6b3bcef3d3570444e46907f90f93651059470cbd4659f8ebaeed1 WatchSource:0}: Error finding container ad64feb6b1a6b3bcef3d3570444e46907f90f93651059470cbd4659f8ebaeed1: Status 404 returned error can't find the container with id ad64feb6b1a6b3bcef3d3570444e46907f90f93651059470cbd4659f8ebaeed1
Oct 29 23:29:40.021021 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:40.020988    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:29:40.021100 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:40.021053    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:29:40.021100 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:40.021083    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:29:40.827896 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:40.827858    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg" event={"ID":"49d459ae-07ae-4c53-aae6-934c25119c6c","Type":"ContainerStarted","Data":"7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18"}
Oct 29 23:29:40.828394 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:40.827908    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg" event={"ID":"49d459ae-07ae-4c53-aae6-934c25119c6c","Type":"ContainerStarted","Data":"ad64feb6b1a6b3bcef3d3570444e46907f90f93651059470cbd4659f8ebaeed1"}
Oct 29 23:29:40.887520 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:40.887460    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg" podStartSLOduration=1.887441548 podStartE2EDuration="1.887441548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:40.885927576 +0000 UTC m=+960.581036455" watchObservedRunningTime="2026-10-29 23:29:40.887441548 +0000 UTC m=+960.582550439"
Oct 29 23:29:41.725764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:41.724810    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:41.725977 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:41.725882    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 29 23:29:41.725977 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:41.725953    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg" podUID="49d459ae-07ae-4c53-aae6-934c25119c6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 29 23:29:42.721763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:42.713875    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 29 23:29:42.721763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:42.713937    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg" podUID="49d459ae-07ae-4c53-aae6-934c25119c6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 29 23:29:43.707101 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:43.707058    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 29 23:29:43.707415 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:43.707385    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg" podUID="49d459ae-07ae-4c53-aae6-934c25119c6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 29 23:29:44.726179 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:44.726142    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 29 23:29:44.726596 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:44.726204    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg" podUID="49d459ae-07ae-4c53-aae6-934c25119c6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 29 23:29:45.721764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:45.707187    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 29 23:29:45.721764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:45.707255    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg" podUID="49d459ae-07ae-4c53-aae6-934c25119c6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 29 23:29:46.929088 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:46.929051    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:46.929560 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:46.929334    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:46.930244 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:46.930221    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:29:53.395868 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.395831    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"]
Oct 29 23:29:53.402176 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.402144    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.422056 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.422029    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"]
Oct 29 23:29:53.431078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.431037    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-b978a0-openshift-default-dockercfg-jmbzw\""
Oct 29 23:29:53.519067 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.519034    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.519223 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.519076    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.519223 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.519167    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tchsm\" (UniqueName: \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-kube-api-access-tchsm\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.519223 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.519211    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.519375 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.519245    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.519375 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.519269    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.519465 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.519387    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.519465 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.519416    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.519465 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.519445    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621042 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621010    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621230 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621075    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621230 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621126    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tchsm\" (UniqueName: \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-kube-api-access-tchsm\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621230 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621169    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621230 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621209    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621463 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621241    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621463 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621302    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621463 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621339    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621463 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621383    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621836 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621811    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621856    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.621923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621861    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.622002 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.621953    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.622104 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.622080    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.623984 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.623959    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.624193 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.624175    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.648944 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.648886    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.653831 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.653806    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tchsm\" (UniqueName: \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-kube-api-access-tchsm\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:53.716591 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:53.716560    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:54.077553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:54.077527    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"]
Oct 29 23:29:54.079766 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:29:54.079710    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe7a7c5a_d8b8_4f9d_86f2_435dc7146711.slice/crio-9fed2c8742727fdbc4127a8ba0dbdfeac637b9841e3a0b1ab5c3bfa5625ac7c7 WatchSource:0}: Error finding container 9fed2c8742727fdbc4127a8ba0dbdfeac637b9841e3a0b1ab5c3bfa5625ac7c7: Status 404 returned error can't find the container with id 9fed2c8742727fdbc4127a8ba0dbdfeac637b9841e3a0b1ab5c3bfa5625ac7c7
Oct 29 23:29:54.082197 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:54.082165    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:29:54.082261 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:54.082228    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:29:54.082261 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:54.082257    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:29:54.930871 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:54.930836    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" event={"ID":"be7a7c5a-d8b8-4f9d-86f2-435dc7146711","Type":"ContainerStarted","Data":"3bc838ab49f5e603fb7f88a45c37be2291e5e047ca91651a5a81a7336219dc74"}
Oct 29 23:29:54.931350 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:54.930894    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" event={"ID":"be7a7c5a-d8b8-4f9d-86f2-435dc7146711","Type":"ContainerStarted","Data":"9fed2c8742727fdbc4127a8ba0dbdfeac637b9841e3a0b1ab5c3bfa5625ac7c7"}
Oct 29 23:29:54.991090 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:54.991024    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" podStartSLOduration=1.990999016 podStartE2EDuration="1.990999016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:54.99021333 +0000 UTC m=+974.685322206" watchObservedRunningTime="2026-10-29 23:29:54.990999016 +0000 UTC m=+974.686107894"
Oct 29 23:29:55.725966 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:55.725926    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:29:55.727549 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:55.727522    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 29 23:29:55.727671 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:55.727568    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 29 23:29:56.725764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:56.723888    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 29 23:29:56.725764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:56.723950    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 29 23:29:57.726165 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:57.726118    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 29 23:29:57.726626 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:57.726173    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 29 23:29:58.725765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:58.725168    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 29 23:29:58.725765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:58.725243    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 29 23:29:59.131340 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:59.131257    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:29:59.717118 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:59.717082    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 29 23:29:59.717370 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:29:59.717344    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 29 23:29:59.783683 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:29:59.783649    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:30:00.206207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.206176    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"]
Oct 29 23:30:00.210274 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.210249    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"
Oct 29 23:30:00.217631 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.217595    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"]
Oct 29 23:30:00.222015 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.221988    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"
Oct 29 23:30:00.236268 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.236225    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:30:00.262794 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.252798    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"]
Oct 29 23:30:00.262794 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.256651    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"]
Oct 29 23:30:00.262794 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.260127    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"]
Oct 29 23:30:00.264651 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.264628    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"
Oct 29 23:30:00.287104 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.287078    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"]
Oct 29 23:30:00.396088 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.396050    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2p54\" (UniqueName: \"kubernetes.io/projected/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-kube-api-access-w2p54\") pod \"maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94\" (UID: \"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"
Oct 29 23:30:00.396538 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.396515    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8853a3df-1182-45aa-99e4-4272aef97e7b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76\" (UID: \"8853a3df-1182-45aa-99e4-4272aef97e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"
Oct 29 23:30:00.396662 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.396649    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbq27\" (UniqueName: \"kubernetes.io/projected/8853a3df-1182-45aa-99e4-4272aef97e7b-kube-api-access-wbq27\") pod \"maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76\" (UID: \"8853a3df-1182-45aa-99e4-4272aef97e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"
Oct 29 23:30:00.396809 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.396795    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94\" (UID: \"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"
Oct 29 23:30:00.396971 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.396955    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxr2q\" (UniqueName: \"kubernetes.io/projected/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-kube-api-access-xxr2q\") pod \"maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r\" (UID: \"23d0bc6c-7c9a-48aa-9722-ccb944f0a912\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"
Oct 29 23:30:00.397076 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.397063    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r\" (UID: \"23d0bc6c-7c9a-48aa-9722-ccb944f0a912\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"
Oct 29 23:30:00.499825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.498387    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8853a3df-1182-45aa-99e4-4272aef97e7b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76\" (UID: \"8853a3df-1182-45aa-99e4-4272aef97e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"
Oct 29 23:30:00.499825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.498452    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbq27\" (UniqueName: \"kubernetes.io/projected/8853a3df-1182-45aa-99e4-4272aef97e7b-kube-api-access-wbq27\") pod \"maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76\" (UID: \"8853a3df-1182-45aa-99e4-4272aef97e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"
Oct 29 23:30:00.499825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.498523    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94\" (UID: \"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"
Oct 29 23:30:00.499825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.498589    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxr2q\" (UniqueName: \"kubernetes.io/projected/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-kube-api-access-xxr2q\") pod \"maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r\" (UID: \"23d0bc6c-7c9a-48aa-9722-ccb944f0a912\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"
Oct 29 23:30:00.499825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.498630    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r\" (UID: \"23d0bc6c-7c9a-48aa-9722-ccb944f0a912\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"
Oct 29 23:30:00.499825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.498684    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2p54\" (UniqueName: \"kubernetes.io/projected/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-kube-api-access-w2p54\") pod \"maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94\" (UID: \"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"
Oct 29 23:30:00.500282 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.500049    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8853a3df-1182-45aa-99e4-4272aef97e7b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76\" (UID: \"8853a3df-1182-45aa-99e4-4272aef97e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"
Oct 29 23:30:00.502815 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.501034    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94\" (UID: \"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"
Oct 29 23:30:00.503923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.503888    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r\" (UID: \"23d0bc6c-7c9a-48aa-9722-ccb944f0a912\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"
Oct 29 23:30:00.541760 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.534689    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbq27\" (UniqueName: \"kubernetes.io/projected/8853a3df-1182-45aa-99e4-4272aef97e7b-kube-api-access-wbq27\") pod \"maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76\" (UID: \"8853a3df-1182-45aa-99e4-4272aef97e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"
Oct 29 23:30:00.548884 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.544907    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"
Oct 29 23:30:00.550082 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.549577    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2p54\" (UniqueName: \"kubernetes.io/projected/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-kube-api-access-w2p54\") pod \"maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94\" (UID: \"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"
Oct 29 23:30:00.592807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.592781    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxr2q\" (UniqueName: \"kubernetes.io/projected/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-kube-api-access-xxr2q\") pod \"maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r\" (UID: \"23d0bc6c-7c9a-48aa-9722-ccb944f0a912\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"
Oct 29 23:30:00.601704 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.601673    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"
Oct 29 23:30:00.833820 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:00.832460    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"
Oct 29 23:30:01.650042 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:01.650004    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"]
Oct 29 23:30:01.732903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:01.732809    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:30:01.732903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:01.732891    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:01.769841 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:01.769801    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"]
Oct 29 23:30:01.900891 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:01.894418    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"]
Oct 29 23:30:02.010473 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:02.010441    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94" event={"ID":"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a","Type":"ContainerStarted","Data":"5ada97f177537088e165671c14abc6e017170f35b2534070d4fbd7159c52ba21"}
Oct 29 23:30:02.012832 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:02.012801    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76" event={"ID":"8853a3df-1182-45aa-99e4-4272aef97e7b","Type":"ContainerStarted","Data":"2feaaa500c5677c4f7b09e7b732dbb3b81b5804dbd99153ef4e7fee8b5ad4579"}
Oct 29 23:30:02.015190 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:02.015164    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r" event={"ID":"23d0bc6c-7c9a-48aa-9722-ccb944f0a912","Type":"ContainerStarted","Data":"61363a3425001ac261d4082395c58f6c425302c7a85899d07276d811674909ab"}
Oct 29 23:30:02.805761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:02.804958    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:30:02.805761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:02.805073    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:03.113198 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:03.113113    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:30:03.114549 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:03.114504    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:30:03.223762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:03.223495    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:30:03.948847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:03.948576    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"]
Oct 29 23:30:03.979913 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:03.979882    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"]
Oct 29 23:30:03.980306 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:03.980286    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:03.999360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:03.998550    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-92f6b3\"/\"distinct-w3-92f6b3-kserve-self-signed-certs\""
Oct 29 23:30:04.167723 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.167683    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d02fd950-ae55-44f5-9fd9-56fa42db262b-tls-certs\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.168618 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.168586    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5tnj\" (UniqueName: \"kubernetes.io/projected/d02fd950-ae55-44f5-9fd9-56fa42db262b-kube-api-access-x5tnj\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.168825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.168668    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-dshm\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.168825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.168720    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-model-cache\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.168825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.168774    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-home\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.168825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.168819    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-tmp-dir\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.272660 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.272572    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d02fd950-ae55-44f5-9fd9-56fa42db262b-tls-certs\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.272858 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.272659    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5tnj\" (UniqueName: \"kubernetes.io/projected/d02fd950-ae55-44f5-9fd9-56fa42db262b-kube-api-access-x5tnj\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.272858 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.272708    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-dshm\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.272858 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.272779    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-model-cache\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.272858 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.272814    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-home\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.273076 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.272856    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-tmp-dir\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.273430 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.273401    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-tmp-dir\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.279485 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.279453    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-model-cache\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.279614 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.279521    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-home\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.286272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.286218    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-dshm\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.288011 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.286636    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d02fd950-ae55-44f5-9fd9-56fa42db262b-tls-certs\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.314099 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.314065    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5tnj\" (UniqueName: \"kubernetes.io/projected/d02fd950-ae55-44f5-9fd9-56fa42db262b-kube-api-access-x5tnj\") pod \"distinct-w3-92f6b3-kserve-769f875d6-tphx6\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.511765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.510184    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"]
Oct 29 23:30:04.515112 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.515092    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:04.538172 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.538142    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"]
Oct 29 23:30:04.538342 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.538325    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:30:04.576500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.576464    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-00b7cd06-lq78c\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:04.576660 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.576520    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-00b7cd06-lq78c\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:04.576660 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.576560    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v5hx\" (UniqueName: \"kubernetes.io/projected/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-kube-api-access-5v5hx\") pod \"maas-api-revoke-sub-00b7cd06-lq78c\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:04.598852 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.598821    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:04.678760 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.678699    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-00b7cd06-lq78c\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:04.678908 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.678775    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-00b7cd06-lq78c\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:04.678908 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.678821    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v5hx\" (UniqueName: \"kubernetes.io/projected/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-kube-api-access-5v5hx\") pod \"maas-api-revoke-sub-00b7cd06-lq78c\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:04.680436 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.680408    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-00b7cd06-lq78c\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:04.686380 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.686322    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-00b7cd06-lq78c\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:04.712754 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.712715    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v5hx\" (UniqueName: \"kubernetes.io/projected/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-kube-api-access-5v5hx\") pod \"maas-api-revoke-sub-00b7cd06-lq78c\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:04.835231 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.835151    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:04.989367 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:04.989332    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"]
Oct 29 23:30:05.023428 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:30:05.023364    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd02fd950_ae55_44f5_9fd9_56fa42db262b.slice/crio-1aa11a5009390b6d3cf6b43006e6ee21c2071371d90be3d17e625c1e6334c83f WatchSource:0}: Error finding container 1aa11a5009390b6d3cf6b43006e6ee21c2071371d90be3d17e625c1e6334c83f: Status 404 returned error can't find the container with id 1aa11a5009390b6d3cf6b43006e6ee21c2071371d90be3d17e625c1e6334c83f
Oct 29 23:30:05.097589 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:05.097557    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" event={"ID":"d02fd950-ae55-44f5-9fd9-56fa42db262b","Type":"ContainerStarted","Data":"1aa11a5009390b6d3cf6b43006e6ee21c2071371d90be3d17e625c1e6334c83f"}
Oct 29 23:30:05.251149 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:30:05.248772    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdbe482f_fa7e_4fe0_bc76_a01df162bf4e.slice/crio-1789eeef87c200042e065b2d78c80f18285788b535382b5bd183a7bf11db8ad6 WatchSource:0}: Error finding container 1789eeef87c200042e065b2d78c80f18285788b535382b5bd183a7bf11db8ad6: Status 404 returned error can't find the container with id 1789eeef87c200042e065b2d78c80f18285788b535382b5bd183a7bf11db8ad6
Oct 29 23:30:05.251149 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:05.249372    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"]
Oct 29 23:30:06.113418 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:06.113380    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c" event={"ID":"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e","Type":"ContainerStarted","Data":"1789eeef87c200042e065b2d78c80f18285788b535382b5bd183a7bf11db8ad6"}
Oct 29 23:30:06.115388 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:06.115358    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" event={"ID":"d02fd950-ae55-44f5-9fd9-56fa42db262b","Type":"ContainerStarted","Data":"dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326"}
Oct 29 23:30:06.168772 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:06.165644    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podStartSLOduration=3.16563106 podStartE2EDuration="3.16563106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:06.163935511 +0000 UTC m=+985.859044392" watchObservedRunningTime="2026-10-29 23:30:06.16563106 +0000 UTC m=+985.860739953"
Oct 29 23:30:06.599694 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:06.599656    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:07.123287 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:07.123030    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94" event={"ID":"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a","Type":"ContainerStarted","Data":"8b9bf2deb27e499cc02f0d237861fe42a5c60f6c3f9d4861e2ef1cbb880a569b"}
Oct 29 23:30:07.171691 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:07.171606    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94" podStartSLOduration=2.20129431 podStartE2EDuration="7.171592171s" totalImagesPullingTime="4.970297861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:07.170533598 +0000 UTC m=+986.865642476" watchObservedRunningTime="2026-10-29 23:30:07.171592171 +0000 UTC m=+986.866701050"
Oct 29 23:30:08.130814 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.130783    6040 generic.go:356] "Generic (PLEG): container finished" podID="cdbe482f-fa7e-4fe0-bc76-a01df162bf4e" containerID="8f2deccd32e43412a6418c6910d77f7babe322cf1ede1a7fb1cba782e43ac075" exitCode=0
Oct 29 23:30:08.131235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.130867    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c" event={"ID":"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e","Type":"ContainerDied","Data":"8f2deccd32e43412a6418c6910d77f7babe322cf1ede1a7fb1cba782e43ac075"}
Oct 29 23:30:08.132463 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.132434    6040 generic.go:356] "Generic (PLEG): container finished" podID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerID="649b5f1b6549d2d9a7d0c071ea723ea28bba59dc9267fbe529c50a86e732e543" exitCode=60
Oct 29 23:30:08.132576 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.132513    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r" event={"ID":"23d0bc6c-7c9a-48aa-9722-ccb944f0a912","Type":"ContainerDied","Data":"649b5f1b6549d2d9a7d0c071ea723ea28bba59dc9267fbe529c50a86e732e543"}
Oct 29 23:30:08.132708 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.132693    6040 scope.go:122] "RemoveContainer" containerID="649b5f1b6549d2d9a7d0c071ea723ea28bba59dc9267fbe529c50a86e732e543"
Oct 29 23:30:08.134181 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.134162    6040 generic.go:356] "Generic (PLEG): container finished" podID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerID="8b9bf2deb27e499cc02f0d237861fe42a5c60f6c3f9d4861e2ef1cbb880a569b" exitCode=60
Oct 29 23:30:08.134284 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.134243    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94" event={"ID":"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a","Type":"ContainerDied","Data":"8b9bf2deb27e499cc02f0d237861fe42a5c60f6c3f9d4861e2ef1cbb880a569b"}
Oct 29 23:30:08.134553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.134537    6040 scope.go:122] "RemoveContainer" containerID="8b9bf2deb27e499cc02f0d237861fe42a5c60f6c3f9d4861e2ef1cbb880a569b"
Oct 29 23:30:08.136623 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.136602    6040 generic.go:356] "Generic (PLEG): container finished" podID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerID="09d214cb10d00e5e244ddd84d0fcb2f45a878c312b21e9d77fa60e3bd2f3abac" exitCode=60
Oct 29 23:30:08.136696 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.136645    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76" event={"ID":"8853a3df-1182-45aa-99e4-4272aef97e7b","Type":"ContainerDied","Data":"09d214cb10d00e5e244ddd84d0fcb2f45a878c312b21e9d77fa60e3bd2f3abac"}
Oct 29 23:30:08.137176 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.136854    6040 scope.go:122] "RemoveContainer" containerID="09d214cb10d00e5e244ddd84d0fcb2f45a878c312b21e9d77fa60e3bd2f3abac"
Oct 29 23:30:08.534708 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.534676    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"]
Oct 29 23:30:08.538654 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.538633    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.560449 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.560117    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-92f6b3-openshift-default-dockercfg-lbv8z\""
Oct 29 23:30:08.563948 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.563915    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"]
Oct 29 23:30:08.631283 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.631245    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.631440 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.631290    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqb7b\" (UniqueName: \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-kube-api-access-zqb7b\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.631440 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.631428    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.631515 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.631455    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.631515 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.631506    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.631579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.631547    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.631612 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.631583    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.631612 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.631606    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.631709 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.631691    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732457 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732383    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732457 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732424    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqb7b\" (UniqueName: \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-kube-api-access-zqb7b\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732480    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732497    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732525    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732555    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732603    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732637    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732980 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732700    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732980 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732824    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732980 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732890    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.732980 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.732906    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.733135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.733004    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.733468 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.733447    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.735035 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.735008    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.735145 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.735130    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.755589 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.755559    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.755589 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.755576    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqb7b\" (UniqueName: \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-kube-api-access-zqb7b\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.841083 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.841050    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"]
Oct 29 23:30:08.846075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.846059    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:08.852136 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.852111    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:08.891576 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.891545    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"]
Oct 29 23:30:08.935720 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.935688    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:08.935899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.935725    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6291cdad-e6f2-461f-be7e-d970959ae576-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:08.935899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.935772    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:08.935899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.935803    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:08.935899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.935844    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:08.935899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.935871    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:08.936141 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.935918    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:08.936141 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.935943    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt2x9\" (UniqueName: \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-kube-api-access-lt2x9\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:08.936141 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:08.936000    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6291cdad-e6f2-461f-be7e-d970959ae576-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.036839 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.036808    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.036961 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.036843    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt2x9\" (UniqueName: \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-kube-api-access-lt2x9\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.036961 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.036889    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6291cdad-e6f2-461f-be7e-d970959ae576-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.037074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.037001    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.037074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.037022    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6291cdad-e6f2-461f-be7e-d970959ae576-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.037074 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.037051    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.037218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.037082    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.037218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.037125    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.037218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.037160    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.037474 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.037419    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.037551 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.037480    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.037606 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.037555    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.037648 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.037603    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.037788 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.037769    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6291cdad-e6f2-461f-be7e-d970959ae576-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.039234 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.039214    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.039503 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.039484    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6291cdad-e6f2-461f-be7e-d970959ae576-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.061028 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.061008    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.061270 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.061249    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt2x9\" (UniqueName: \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-kube-api-access-lt2x9\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.146465 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.146425    6040 generic.go:356] "Generic (PLEG): container finished" podID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerID="5f163c52b947c1015db8f18cf38f88afc83346e457f13bea3b3f16b038f8130c" exitCode=60
Oct 29 23:30:09.147022 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.146526    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r" event={"ID":"23d0bc6c-7c9a-48aa-9722-ccb944f0a912","Type":"ContainerDied","Data":"5f163c52b947c1015db8f18cf38f88afc83346e457f13bea3b3f16b038f8130c"}
Oct 29 23:30:09.147022 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.146586    6040 scope.go:122] "RemoveContainer" containerID="649b5f1b6549d2d9a7d0c071ea723ea28bba59dc9267fbe529c50a86e732e543"
Oct 29 23:30:09.147022 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.146929    6040 scope.go:122] "RemoveContainer" containerID="5f163c52b947c1015db8f18cf38f88afc83346e457f13bea3b3f16b038f8130c"
Oct 29 23:30:09.147255 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:30:09.147227    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r_odh-ai-gateway-infra(23d0bc6c-7c9a-48aa-9722-ccb944f0a912)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912"
Oct 29 23:30:09.148975 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.148948    6040 generic.go:356] "Generic (PLEG): container finished" podID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerID="09ae0da384714bbfab1faad3959d60d7d786cc5506fbab5a530454dcd8fb684d" exitCode=60
Oct 29 23:30:09.149089 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.149023    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94" event={"ID":"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a","Type":"ContainerDied","Data":"09ae0da384714bbfab1faad3959d60d7d786cc5506fbab5a530454dcd8fb684d"}
Oct 29 23:30:09.149268 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.149251    6040 scope.go:122] "RemoveContainer" containerID="09ae0da384714bbfab1faad3959d60d7d786cc5506fbab5a530454dcd8fb684d"
Oct 29 23:30:09.149482 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:30:09.149466    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94_odh-ai-gateway-infra(d7d85ae4-5b64-4a8b-8303-ceb88e55b82a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a"
Oct 29 23:30:09.151117 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.151095    6040 generic.go:356] "Generic (PLEG): container finished" podID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerID="92b20b73bff41430b379430f3989643218fe620e653aa446ff95695cb4ce4f6f" exitCode=60
Oct 29 23:30:09.151195 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.151158    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76" event={"ID":"8853a3df-1182-45aa-99e4-4272aef97e7b","Type":"ContainerDied","Data":"92b20b73bff41430b379430f3989643218fe620e653aa446ff95695cb4ce4f6f"}
Oct 29 23:30:09.151507 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.151398    6040 scope.go:122] "RemoveContainer" containerID="92b20b73bff41430b379430f3989643218fe620e653aa446ff95695cb4ce4f6f"
Oct 29 23:30:09.151603 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:30:09.151584    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76_odh-ai-gateway-infra(8853a3df-1182-45aa-99e4-4272aef97e7b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b"
Oct 29 23:30:09.159651 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.159629    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:09.160285 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.160262    6040 scope.go:122] "RemoveContainer" containerID="8b9bf2deb27e499cc02f0d237861fe42a5c60f6c3f9d4861e2ef1cbb880a569b"
Oct 29 23:30:09.178919 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.178889    6040 scope.go:122] "RemoveContainer" containerID="09d214cb10d00e5e244ddd84d0fcb2f45a878c312b21e9d77fa60e3bd2f3abac"
Oct 29 23:30:09.179219 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.179193    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"]
Oct 29 23:30:09.182772 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:30:09.182734    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd5e5a9_6c82_453e_9be5_dfc9df17a6e2.slice/crio-e931b3b2186e2f71ccfba07bd8e42ba281641a12932ceed2969644f44b1a55a7 WatchSource:0}: Error finding container e931b3b2186e2f71ccfba07bd8e42ba281641a12932ceed2969644f44b1a55a7: Status 404 returned error can't find the container with id e931b3b2186e2f71ccfba07bd8e42ba281641a12932ceed2969644f44b1a55a7
Oct 29 23:30:09.186040 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.186002    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:09.186127 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.186091    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:09.186183 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.186135    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:09.307193 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.307030    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:09.340718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.340685    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-xks-ca\") pod \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") "
Oct 29 23:30:09.340868 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.340808    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-openshift-ca\") pod \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") "
Oct 29 23:30:09.340928 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.340903    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-kube-api-access-5v5hx\" (UniqueName: \"kubernetes.io/projected/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-kube-api-access-5v5hx\") pod \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\" (UID: \"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e\") "
Oct 29 23:30:09.341143 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.341119    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-openshift-ca" pod "cdbe482f-fa7e-4fe0-bc76-a01df162bf4e" (UID: "cdbe482f-fa7e-4fe0-bc76-a01df162bf4e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:09.341446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.341423    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:09.342966 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.342938    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-xks-ca" pod "cdbe482f-fa7e-4fe0-bc76-a01df162bf4e" (UID: "cdbe482f-fa7e-4fe0-bc76-a01df162bf4e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:09.343066 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.342996    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-kube-api-access-5v5hx" pod "cdbe482f-fa7e-4fe0-bc76-a01df162bf4e" (UID: "cdbe482f-fa7e-4fe0-bc76-a01df162bf4e"). InnerVolumeSpecName "kube-api-access-5v5hx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:09.442242 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.442209    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v5hx\" (UniqueName: \"kubernetes.io/projected/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-kube-api-access-5v5hx\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:09.442242 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.442237    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:09.476988 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.476956    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"]
Oct 29 23:30:09.477211 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:30:09.477188    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6291cdad_e6f2_461f_be7e_d970959ae576.slice/crio-60f9a308fd2b910eef238c222d98406c961902b8a5fc4241c95dc9125ecf7389 WatchSource:0}: Error finding container 60f9a308fd2b910eef238c222d98406c961902b8a5fc4241c95dc9125ecf7389: Status 404 returned error can't find the container with id 60f9a308fd2b910eef238c222d98406c961902b8a5fc4241c95dc9125ecf7389
Oct 29 23:30:09.479910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.479873    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:09.480000 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.479959    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:09.480044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:09.480002    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:10.160864 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.160829    6040 scope.go:122] "RemoveContainer" containerID="92b20b73bff41430b379430f3989643218fe620e653aa446ff95695cb4ce4f6f"
Oct 29 23:30:10.161318 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:30:10.161127    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76_odh-ai-gateway-infra(8853a3df-1182-45aa-99e4-4272aef97e7b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b"
Oct 29 23:30:10.161787 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.161765    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"
Oct 29 23:30:10.161889 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.161764    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c" event={"ID":"cdbe482f-fa7e-4fe0-bc76-a01df162bf4e","Type":"ContainerDied","Data":"1789eeef87c200042e065b2d78c80f18285788b535382b5bd183a7bf11db8ad6"}
Oct 29 23:30:10.161949 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.161894    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1789eeef87c200042e065b2d78c80f18285788b535382b5bd183a7bf11db8ad6"
Oct 29 23:30:10.164266 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.164240    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952" event={"ID":"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2","Type":"ContainerStarted","Data":"a944855ff49c57664a190b8a4324e2cc8e65fbfce29ec81cedf163fafeb42685"}
Oct 29 23:30:10.164360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.164290    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952" event={"ID":"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2","Type":"ContainerStarted","Data":"e931b3b2186e2f71ccfba07bd8e42ba281641a12932ceed2969644f44b1a55a7"}
Oct 29 23:30:10.165984 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.165966    6040 scope.go:122] "RemoveContainer" containerID="5f163c52b947c1015db8f18cf38f88afc83346e457f13bea3b3f16b038f8130c"
Oct 29 23:30:10.166366 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:30:10.166343    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r_odh-ai-gateway-infra(23d0bc6c-7c9a-48aa-9722-ccb944f0a912)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912"
Oct 29 23:30:10.167899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.167840    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" event={"ID":"6291cdad-e6f2-461f-be7e-d970959ae576","Type":"ContainerStarted","Data":"a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264"}
Oct 29 23:30:10.167899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.167877    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" event={"ID":"6291cdad-e6f2-461f-be7e-d970959ae576","Type":"ContainerStarted","Data":"60f9a308fd2b910eef238c222d98406c961902b8a5fc4241c95dc9125ecf7389"}
Oct 29 23:30:10.169967 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.169951    6040 scope.go:122] "RemoveContainer" containerID="09ae0da384714bbfab1faad3959d60d7d786cc5506fbab5a530454dcd8fb684d"
Oct 29 23:30:10.170165 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:30:10.170147    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94_odh-ai-gateway-infra(d7d85ae4-5b64-4a8b-8303-ceb88e55b82a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a"
Oct 29 23:30:10.331417 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.331353    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952" podStartSLOduration=2.331332834 podStartE2EDuration="2.331332834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:10.330044129 +0000 UTC m=+990.025153008" watchObservedRunningTime="2026-10-29 23:30:10.331332834 +0000 UTC m=+990.026441715"
Oct 29 23:30:10.351797 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.333197    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" podStartSLOduration=2.333184624 podStartE2EDuration="2.333184624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:10.257413818 +0000 UTC m=+989.952522697" watchObservedRunningTime="2026-10-29 23:30:10.333184624 +0000 UTC m=+990.028293504"
Oct 29 23:30:10.853808 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.853347    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:10.855146 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.855120    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 29 23:30:10.855246 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:10.855179    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952" podUID="ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 29 23:30:11.160776 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.160732    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:11.162842 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.162814    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 29 23:30:11.163262 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.162883    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" podUID="6291cdad-e6f2-461f-be7e-d970959ae576" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 29 23:30:11.273768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.273725    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"]
Oct 29 23:30:11.274224 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.274210    6040 cpu_manager.go:414] "containerMap: removing container" podUID="cdbe482f-fa7e-4fe0-bc76-a01df162bf4e" containerName="revoke-keys"
Oct 29 23:30:11.274276 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.274226    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cdbe482f-fa7e-4fe0-bc76-a01df162bf4e" containerName="revoke-keys"
Oct 29 23:30:11.274322 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.274313    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="cdbe482f-fa7e-4fe0-bc76-a01df162bf4e" containerName="revoke-keys"
Oct 29 23:30:11.277755 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.277725    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:11.294654 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.294632    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:30:11.298001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.297976    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"]
Oct 29 23:30:11.373053 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.373008    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxghn\" (UniqueName: \"kubernetes.io/projected/3895e845-9690-47db-89a0-0c716657236a-kube-api-access-nxghn\") pod \"maas-api-revoke-sub-347c0d86-bwnht\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:11.373053 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.373045    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3895e845-9690-47db-89a0-0c716657236a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-347c0d86-bwnht\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:11.373310 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.373105    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3895e845-9690-47db-89a0-0c716657236a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-347c0d86-bwnht\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:11.474710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.474624    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3895e845-9690-47db-89a0-0c716657236a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-347c0d86-bwnht\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:11.474874 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.474811    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxghn\" (UniqueName: \"kubernetes.io/projected/3895e845-9690-47db-89a0-0c716657236a-kube-api-access-nxghn\") pod \"maas-api-revoke-sub-347c0d86-bwnht\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:11.474874 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.474842    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3895e845-9690-47db-89a0-0c716657236a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-347c0d86-bwnht\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:11.475658 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.475635    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3895e845-9690-47db-89a0-0c716657236a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-347c0d86-bwnht\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:11.477687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.477660    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3895e845-9690-47db-89a0-0c716657236a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-347c0d86-bwnht\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:11.498581 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.498557    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxghn\" (UniqueName: \"kubernetes.io/projected/3895e845-9690-47db-89a0-0c716657236a-kube-api-access-nxghn\") pod \"maas-api-revoke-sub-347c0d86-bwnht\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:11.591228 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.591195    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:11.853808 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.853255    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 29 23:30:11.853808 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.853308    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952" podUID="ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 29 23:30:11.951761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:11.951534    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"]
Oct 29 23:30:11.956355 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:30:11.956324    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3895e845_9690_47db_89a0_0c716657236a.slice/crio-62da7baa2e724331e5329df7aa71ac59771a75f7e6e1d6cc36ce2ea8ac57be6a WatchSource:0}: Error finding container 62da7baa2e724331e5329df7aa71ac59771a75f7e6e1d6cc36ce2ea8ac57be6a: Status 404 returned error can't find the container with id 62da7baa2e724331e5329df7aa71ac59771a75f7e6e1d6cc36ce2ea8ac57be6a
Oct 29 23:30:12.160975 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:12.160941    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 29 23:30:12.161130 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:12.160992    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" podUID="6291cdad-e6f2-461f-be7e-d970959ae576" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 29 23:30:12.183050 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:12.182901    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht" event={"ID":"3895e845-9690-47db-89a0-0c716657236a","Type":"ContainerStarted","Data":"62da7baa2e724331e5329df7aa71ac59771a75f7e6e1d6cc36ce2ea8ac57be6a"}
Oct 29 23:30:12.853195 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:12.853167    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 29 23:30:12.853433 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:12.853413    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952" podUID="ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 29 23:30:13.160014 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:13.159982    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 29 23:30:13.160184 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:13.160029    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" podUID="6291cdad-e6f2-461f-be7e-d970959ae576" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 29 23:30:13.190424 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:13.190326    6040 generic.go:356] "Generic (PLEG): container finished" podID="3895e845-9690-47db-89a0-0c716657236a" containerID="6427fca2af1ff5667e1e1671c998dbf7a67532e2c32a2da1fee5652a8e7cd802" exitCode=0
Oct 29 23:30:13.190424 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:13.190363    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht" event={"ID":"3895e845-9690-47db-89a0-0c716657236a","Type":"ContainerDied","Data":"6427fca2af1ff5667e1e1671c998dbf7a67532e2c32a2da1fee5652a8e7cd802"}
Oct 29 23:30:13.853432 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:13.853393    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 29 23:30:13.853727 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:13.853696    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952" podUID="ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 29 23:30:14.160635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.160600    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 29 23:30:14.160836 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.160662    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" podUID="6291cdad-e6f2-461f-be7e-d970959ae576" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 29 23:30:14.492524 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.492498    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:14.599796 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.599764    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:14.654319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.654270    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3895e845-9690-47db-89a0-0c716657236a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3895e845-9690-47db-89a0-0c716657236a-maas-api-xks-ca\") pod \"3895e845-9690-47db-89a0-0c716657236a\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") "
Oct 29 23:30:14.654575 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.654557    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3895e845-9690-47db-89a0-0c716657236a-kube-api-access-nxghn\" (UniqueName: \"kubernetes.io/projected/3895e845-9690-47db-89a0-0c716657236a-kube-api-access-nxghn\") pod \"3895e845-9690-47db-89a0-0c716657236a\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") "
Oct 29 23:30:14.654853 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.654824    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3895e845-9690-47db-89a0-0c716657236a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3895e845-9690-47db-89a0-0c716657236a-maas-api-openshift-ca\") pod \"3895e845-9690-47db-89a0-0c716657236a\" (UID: \"3895e845-9690-47db-89a0-0c716657236a\") "
Oct 29 23:30:14.655825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.655790    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3895e845-9690-47db-89a0-0c716657236a-maas-api-openshift-ca" pod "3895e845-9690-47db-89a0-0c716657236a" (UID: "3895e845-9690-47db-89a0-0c716657236a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:14.657841 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.657808    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3895e845-9690-47db-89a0-0c716657236a-kube-api-access-nxghn" pod "3895e845-9690-47db-89a0-0c716657236a" (UID: "3895e845-9690-47db-89a0-0c716657236a"). InnerVolumeSpecName "kube-api-access-nxghn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:14.658225 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.658193    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3895e845-9690-47db-89a0-0c716657236a-maas-api-xks-ca" pod "3895e845-9690-47db-89a0-0c716657236a" (UID: "3895e845-9690-47db-89a0-0c716657236a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:14.666043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.666025    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:14.757202 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.756693    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3895e845-9690-47db-89a0-0c716657236a-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:14.757202 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.756723    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxghn\" (UniqueName: \"kubernetes.io/projected/3895e845-9690-47db-89a0-0c716657236a-kube-api-access-nxghn\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:14.757202 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.756766    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3895e845-9690-47db-89a0-0c716657236a-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:14.853223 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.853180    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 29 23:30:14.853424 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:14.853230    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952" podUID="ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 29 23:30:15.160007 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:15.159968    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 29 23:30:15.160202 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:15.160019    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" podUID="6291cdad-e6f2-461f-be7e-d970959ae576" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 29 23:30:15.229761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:15.226436    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"
Oct 29 23:30:15.229957 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:15.229795    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht" event={"ID":"3895e845-9690-47db-89a0-0c716657236a","Type":"ContainerDied","Data":"62da7baa2e724331e5329df7aa71ac59771a75f7e6e1d6cc36ce2ea8ac57be6a"}
Oct 29 23:30:15.229957 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:15.229853    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62da7baa2e724331e5329df7aa71ac59771a75f7e6e1d6cc36ce2ea8ac57be6a"
Oct 29 23:30:15.243925 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:15.243898    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:30:16.061234 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:16.061197    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:16.064607 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:16.062664    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:16.064607 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:16.063700    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:30:16.344300 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:16.344217    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:16.346077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:16.345993    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:16.347574 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:16.347551    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:30:20.423768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:20.423708    6040 scope.go:122] "RemoveContainer" containerID="09ae0da384714bbfab1faad3959d60d7d786cc5506fbab5a530454dcd8fb684d"
Oct 29 23:30:21.369643 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:21.369611    6040 generic.go:356] "Generic (PLEG): container finished" podID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerID="23c4767aaa5f679b2fbc7c31d4fc9455ce58de7d62235f52f5f0b65972b79470" exitCode=60
Oct 29 23:30:21.369833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:21.369685    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94" event={"ID":"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a","Type":"ContainerDied","Data":"23c4767aaa5f679b2fbc7c31d4fc9455ce58de7d62235f52f5f0b65972b79470"}
Oct 29 23:30:21.369833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:21.369770    6040 scope.go:122] "RemoveContainer" containerID="09ae0da384714bbfab1faad3959d60d7d786cc5506fbab5a530454dcd8fb684d"
Oct 29 23:30:21.370142 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:21.370120    6040 scope.go:122] "RemoveContainer" containerID="23c4767aaa5f679b2fbc7c31d4fc9455ce58de7d62235f52f5f0b65972b79470"
Oct 29 23:30:21.370437 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:30:21.370414    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94_odh-ai-gateway-infra(d7d85ae4-5b64-4a8b-8303-ceb88e55b82a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a"
Oct 29 23:30:21.418858 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:21.418837    6040 scope.go:122] "RemoveContainer" containerID="92b20b73bff41430b379430f3989643218fe620e653aa446ff95695cb4ce4f6f"
Oct 29 23:30:22.379833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.379800    6040 generic.go:356] "Generic (PLEG): container finished" podID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerID="a3b984cf701acb611a7ef016c577debfe44d73d96ce0c2447e6d157327d22f9f" exitCode=60
Oct 29 23:30:22.380295 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.379871    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76" event={"ID":"8853a3df-1182-45aa-99e4-4272aef97e7b","Type":"ContainerDied","Data":"a3b984cf701acb611a7ef016c577debfe44d73d96ce0c2447e6d157327d22f9f"}
Oct 29 23:30:22.380295 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.379978    6040 scope.go:122] "RemoveContainer" containerID="92b20b73bff41430b379430f3989643218fe620e653aa446ff95695cb4ce4f6f"
Oct 29 23:30:22.380425 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.380351    6040 scope.go:122] "RemoveContainer" containerID="a3b984cf701acb611a7ef016c577debfe44d73d96ce0c2447e6d157327d22f9f"
Oct 29 23:30:22.380671 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:30:22.380649    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76_odh-ai-gateway-infra(8853a3df-1182-45aa-99e4-4272aef97e7b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b"
Oct 29 23:30:22.425789 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.425733    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"]
Oct 29 23:30:22.581385 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.581356    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"
Oct 29 23:30:22.695651 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.695624    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-maas-api-service-ca\") pod \"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a\" (UID: \"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a\") "
Oct 29 23:30:22.695847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.695823    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-kube-api-access-w2p54\" (UniqueName: \"kubernetes.io/projected/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-kube-api-access-w2p54\") pod \"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a\" (UID: \"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a\") "
Oct 29 23:30:22.695983 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.695959    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-maas-api-service-ca" pod "d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" (UID: "d7d85ae4-5b64-4a8b-8303-ceb88e55b82a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:22.696160 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.696145    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-maas-api-service-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:22.697921 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.697894    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-kube-api-access-w2p54" pod "d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" (UID: "d7d85ae4-5b64-4a8b-8303-ceb88e55b82a"). InnerVolumeSpecName "kube-api-access-w2p54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:22.797040 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:22.796956    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2p54\" (UniqueName: \"kubernetes.io/projected/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a-kube-api-access-w2p54\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:23.387628 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.387597    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"
Oct 29 23:30:23.388125 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.387631    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94" event={"ID":"d7d85ae4-5b64-4a8b-8303-ceb88e55b82a","Type":"ContainerDied","Data":"5ada97f177537088e165671c14abc6e017170f35b2534070d4fbd7159c52ba21"}
Oct 29 23:30:23.388125 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.387703    6040 scope.go:122] "RemoveContainer" containerID="23c4767aaa5f679b2fbc7c31d4fc9455ce58de7d62235f52f5f0b65972b79470"
Oct 29 23:30:23.418419 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.418395    6040 scope.go:122] "RemoveContainer" containerID="5f163c52b947c1015db8f18cf38f88afc83346e457f13bea3b3f16b038f8130c"
Oct 29 23:30:23.436106 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.436080    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"]
Oct 29 23:30:23.443443 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.443417    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"]
Oct 29 23:30:23.453482 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.453457    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b978a0-29888610-2gk94"]
Oct 29 23:30:23.581049 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.581022    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"
Oct 29 23:30:23.706402 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.706374    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8853a3df-1182-45aa-99e4-4272aef97e7b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8853a3df-1182-45aa-99e4-4272aef97e7b-maas-api-service-ca\") pod \"8853a3df-1182-45aa-99e4-4272aef97e7b\" (UID: \"8853a3df-1182-45aa-99e4-4272aef97e7b\") "
Oct 29 23:30:23.706611 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.706594    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8853a3df-1182-45aa-99e4-4272aef97e7b-kube-api-access-wbq27\" (UniqueName: \"kubernetes.io/projected/8853a3df-1182-45aa-99e4-4272aef97e7b-kube-api-access-wbq27\") pod \"8853a3df-1182-45aa-99e4-4272aef97e7b\" (UID: \"8853a3df-1182-45aa-99e4-4272aef97e7b\") "
Oct 29 23:30:23.706780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.706749    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8853a3df-1182-45aa-99e4-4272aef97e7b-maas-api-service-ca" pod "8853a3df-1182-45aa-99e4-4272aef97e7b" (UID: "8853a3df-1182-45aa-99e4-4272aef97e7b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:23.707052 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.707033    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8853a3df-1182-45aa-99e4-4272aef97e7b-maas-api-service-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:23.708910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.708870    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8853a3df-1182-45aa-99e4-4272aef97e7b-kube-api-access-wbq27" pod "8853a3df-1182-45aa-99e4-4272aef97e7b" (UID: "8853a3df-1182-45aa-99e4-4272aef97e7b"). InnerVolumeSpecName "kube-api-access-wbq27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:23.808198 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:23.808159    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbq27\" (UniqueName: \"kubernetes.io/projected/8853a3df-1182-45aa-99e4-4272aef97e7b-kube-api-access-wbq27\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:24.051395 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051317    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"]
Oct 29 23:30:24.051811 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051797    6040 cpu_manager.go:414] "containerMap: removing container" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerName="cleanup"
Oct 29 23:30:24.051865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051812    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerName="cleanup"
Oct 29 23:30:24.051865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051829    6040 cpu_manager.go:414] "containerMap: removing container" podUID="3895e845-9690-47db-89a0-0c716657236a" containerName="revoke-keys"
Oct 29 23:30:24.051865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051836    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3895e845-9690-47db-89a0-0c716657236a" containerName="revoke-keys"
Oct 29 23:30:24.051865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051846    6040 cpu_manager.go:414] "containerMap: removing container" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerName="cleanup"
Oct 29 23:30:24.051865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051851    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerName="cleanup"
Oct 29 23:30:24.051865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051859    6040 cpu_manager.go:414] "containerMap: removing container" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerName="cleanup"
Oct 29 23:30:24.051865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051864    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerName="cleanup"
Oct 29 23:30:24.052073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051873    6040 cpu_manager.go:414] "containerMap: removing container" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerName="cleanup"
Oct 29 23:30:24.052073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051877    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerName="cleanup"
Oct 29 23:30:24.052073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051896    6040 cpu_manager.go:414] "containerMap: removing container" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerName="cleanup"
Oct 29 23:30:24.052073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051903    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerName="cleanup"
Oct 29 23:30:24.052073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.051993    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerName="cleanup"
Oct 29 23:30:24.052073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.052000    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerName="cleanup"
Oct 29 23:30:24.052073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.052006    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerName="cleanup"
Oct 29 23:30:24.052073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.052014    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerName="cleanup"
Oct 29 23:30:24.052073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.052023    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="3895e845-9690-47db-89a0-0c716657236a" containerName="revoke-keys"
Oct 29 23:30:24.055376 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.055359    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.070892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.070872    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-aa7e96-openshift-default-dockercfg-8q4lr\""
Oct 29 23:30:24.078048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.078024    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"]
Oct 29 23:30:24.087564 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.087536    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"]
Oct 29 23:30:24.088280 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.088259    6040 cpu_manager.go:414] "containerMap: removing container" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerName="cleanup"
Oct 29 23:30:24.088322 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.088284    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerName="cleanup"
Oct 29 23:30:24.088410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.088399    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" containerName="cleanup"
Oct 29 23:30:24.088444 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.088419    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b" containerName="cleanup"
Oct 29 23:30:24.092092 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.092074    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.111672 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.111638    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.111856 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.111719    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22a149b5-4e2b-44b1-b454-5244a760273e-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.111856 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.111824    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22a149b5-4e2b-44b1-b454-5244a760273e-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.112034 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.111893    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.112034 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.111921    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.112034 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.112008    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.112261 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.112061    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2hz6\" (UniqueName: \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-kube-api-access-t2hz6\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.112261 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.112121    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.112261 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.112156    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.112487 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.112366    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"]
Oct 29 23:30:24.171417 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.171388    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-ae1784-openshift-default-dockercfg-bf8gx\""
Oct 29 23:30:24.213255 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213220    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.213436 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213264    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.213436 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213289    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.213436 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213311    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc61fe28-ed42-4655-99de-d63d17a30a3f-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.213436 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213398    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.213673 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213438    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.213673 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213495    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.213673 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213545    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.213673 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213567    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22a149b5-4e2b-44b1-b454-5244a760273e-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.213673 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213628    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znt72\" (UniqueName: \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-kube-api-access-znt72\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.213673 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213659    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22a149b5-4e2b-44b1-b454-5244a760273e-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.214009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213698    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.214009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213782    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.214009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213823    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.214009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213829    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.214009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213850    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.214009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213922    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.214009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.213983    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.214333 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.214039    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.214333 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.214072    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2hz6\" (UniqueName: \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-kube-api-access-t2hz6\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.214333 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.214185    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.214494 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.214373    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22a149b5-4e2b-44b1-b454-5244a760273e-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.214658 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.214634    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.216510 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.216486    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.216615 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.216514    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22a149b5-4e2b-44b1-b454-5244a760273e-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.237255 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.237224    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.237409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.237390    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2hz6\" (UniqueName: \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-kube-api-access-t2hz6\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.315051 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315012    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315171 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315086    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315236 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315180    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315236 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315204    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc61fe28-ed42-4655-99de-d63d17a30a3f-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315348 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315234    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315348 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315288    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315348 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315331    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315506 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315355    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315506 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315392    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znt72\" (UniqueName: \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-kube-api-access-znt72\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315506 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315430    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315506 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315447    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315506 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315496    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315822 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315803    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.315913 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.315886    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc61fe28-ed42-4655-99de-d63d17a30a3f-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.317613 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.317588    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.317862 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.317845    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.361522 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.361481    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.361674 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.361542    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znt72\" (UniqueName: \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-kube-api-access-znt72\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.369547 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.369520    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:24.406065 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.406027    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:24.407252 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.407229    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"
Oct 29 23:30:24.407393 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.407328    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76" event={"ID":"8853a3df-1182-45aa-99e4-4272aef97e7b","Type":"ContainerDied","Data":"2feaaa500c5677c4f7b09e7b732dbb3b81b5804dbd99153ef4e7fee8b5ad4579"}
Oct 29 23:30:24.407462 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.407428    6040 scope.go:122] "RemoveContainer" containerID="a3b984cf701acb611a7ef016c577debfe44d73d96ce0c2447e6d157327d22f9f"
Oct 29 23:30:24.412092 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.412064    6040 generic.go:356] "Generic (PLEG): container finished" podID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerID="3f220cf983ca1ce4cf8eea0009ef55e72e024329c4a2f224d3309ace98339cd6" exitCode=60
Oct 29 23:30:24.412207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.412138    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r" event={"ID":"23d0bc6c-7c9a-48aa-9722-ccb944f0a912","Type":"ContainerDied","Data":"3f220cf983ca1ce4cf8eea0009ef55e72e024329c4a2f224d3309ace98339cd6"}
Oct 29 23:30:24.412591 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.412573    6040 scope.go:122] "RemoveContainer" containerID="3f220cf983ca1ce4cf8eea0009ef55e72e024329c4a2f224d3309ace98339cd6"
Oct 29 23:30:24.412921 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:30:24.412899    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r_odh-ai-gateway-infra(23d0bc6c-7c9a-48aa-9722-ccb944f0a912)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912"
Oct 29 23:30:24.426842 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.426818    6040 scope.go:122] "RemoveContainer" containerID="5f163c52b947c1015db8f18cf38f88afc83346e457f13bea3b3f16b038f8130c"
Oct 29 23:30:24.427909 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.427886    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7d85ae4-5b64-4a8b-8303-ceb88e55b82a" path="/var/lib/kubelet/pods/d7d85ae4-5b64-4a8b-8303-ceb88e55b82a/volumes"
Oct 29 23:30:24.474716 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.474632    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"]
Oct 29 23:30:24.480870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.480843    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-53c44e-29888610-vrp76"]
Oct 29 23:30:24.569722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.569639    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"]
Oct 29 23:30:24.575706 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.575680    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.598677 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.598648    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"]
Oct 29 23:30:24.618980 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.618944    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9br4\" (UniqueName: \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-kube-api-access-s9br4\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.619150 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.619072    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.619218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.619196    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.619263 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.619249    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.619427 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.619410    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45f800d-3877-415d-b832-92956ed12f88-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.619470 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.619446    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.619507 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.619490    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.619607 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.619586    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45f800d-3877-415d-b832-92956ed12f88-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.619669 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.619636    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.705458 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.705416    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"]
Oct 29 23:30:24.723597 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.723546    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.723966 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.723937    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.724136 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.724109    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.724265 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.724228    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45f800d-3877-415d-b832-92956ed12f88-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.724326 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.724262    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.724326 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.724305    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.724446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.724345    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45f800d-3877-415d-b832-92956ed12f88-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.724446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.724385    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.724544 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.724473    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9br4\" (UniqueName: \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-kube-api-access-s9br4\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.725225 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.725203    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.732091 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.726221    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.732211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.730137    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.732292 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.731687    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.732391 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.731912    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.732496 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.732024    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45f800d-3877-415d-b832-92956ed12f88-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.733054 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:30:24.733032    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a149b5_4e2b_44b1_b454_5244a760273e.slice/crio-6563d1b90d3d445ae413a52ea1ea539c86da29ff3aa463921fffb348e3d4f535 WatchSource:0}: Error finding container 6563d1b90d3d445ae413a52ea1ea539c86da29ff3aa463921fffb348e3d4f535: Status 404 returned error can't find the container with id 6563d1b90d3d445ae413a52ea1ea539c86da29ff3aa463921fffb348e3d4f535
Oct 29 23:30:24.735049 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.735027    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45f800d-3877-415d-b832-92956ed12f88-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.746307 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.746273    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:24.746483 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.746462    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:24.746557 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.746514    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:24.753213 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.753193    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9br4\" (UniqueName: \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-kube-api-access-s9br4\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.754223 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.754197    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:24.777523 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.777499    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"]
Oct 29 23:30:24.782275 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:30:24.782251    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc61fe28_ed42_4655_99de_d63d17a30a3f.slice/crio-02bf775635b2dbba634f53a3a3fe2b037f8dac3610b0b54d29803cd6606fa705 WatchSource:0}: Error finding container 02bf775635b2dbba634f53a3a3fe2b037f8dac3610b0b54d29803cd6606fa705: Status 404 returned error can't find the container with id 02bf775635b2dbba634f53a3a3fe2b037f8dac3610b0b54d29803cd6606fa705
Oct 29 23:30:24.785250 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.785217    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:24.785368 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.785295    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:24.785368 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.785343    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:24.900876 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:24.900608    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:25.215933 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.215901    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"]
Oct 29 23:30:25.217108 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:30:25.216803    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45f800d_3877_415d_b832_92956ed12f88.slice/crio-7357ee653daa4cec10663757f2354721d9c2733710b1b5697d950a28f3f4ccb3 WatchSource:0}: Error finding container 7357ee653daa4cec10663757f2354721d9c2733710b1b5697d950a28f3f4ccb3: Status 404 returned error can't find the container with id 7357ee653daa4cec10663757f2354721d9c2733710b1b5697d950a28f3f4ccb3
Oct 29 23:30:25.219351 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.219321    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:25.219496 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.219384    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:25.219496 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.219413    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:25.436826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.436263    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" event={"ID":"cc61fe28-ed42-4655-99de-d63d17a30a3f","Type":"ContainerStarted","Data":"beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e"}
Oct 29 23:30:25.436826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.436316    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" event={"ID":"cc61fe28-ed42-4655-99de-d63d17a30a3f","Type":"ContainerStarted","Data":"02bf775635b2dbba634f53a3a3fe2b037f8dac3610b0b54d29803cd6606fa705"}
Oct 29 23:30:25.439304 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.439272    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" event={"ID":"f45f800d-3877-415d-b832-92956ed12f88","Type":"ContainerStarted","Data":"18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3"}
Oct 29 23:30:25.439440 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.439325    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" event={"ID":"f45f800d-3877-415d-b832-92956ed12f88","Type":"ContainerStarted","Data":"7357ee653daa4cec10663757f2354721d9c2733710b1b5697d950a28f3f4ccb3"}
Oct 29 23:30:25.443349 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.443324    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" event={"ID":"22a149b5-4e2b-44b1-b454-5244a760273e","Type":"ContainerStarted","Data":"bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb"}
Oct 29 23:30:25.443433 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.443373    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" event={"ID":"22a149b5-4e2b-44b1-b454-5244a760273e","Type":"ContainerStarted","Data":"6563d1b90d3d445ae413a52ea1ea539c86da29ff3aa463921fffb348e3d4f535"}
Oct 29 23:30:25.493956 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.493709    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" podStartSLOduration=1.493689428 podStartE2EDuration="1.493689428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:25.490696944 +0000 UTC m=+1005.185805829" watchObservedRunningTime="2026-10-29 23:30:25.493689428 +0000 UTC m=+1005.188798307"
Oct 29 23:30:25.561784 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.559557    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" podStartSLOduration=1.55953713 podStartE2EDuration="1.55953713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:25.553191327 +0000 UTC m=+1005.248300209" watchObservedRunningTime="2026-10-29 23:30:25.55953713 +0000 UTC m=+1005.254646011"
Oct 29 23:30:25.569826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.569790    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"]
Oct 29 23:30:25.611729 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.611102    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" podStartSLOduration=1.6110810230000001 podStartE2EDuration="1.611081023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:25.609786558 +0000 UTC m=+1005.304895440" watchObservedRunningTime="2026-10-29 23:30:25.611081023 +0000 UTC m=+1005.306189903"
Oct 29 23:30:25.901682 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.901653    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:25.908144 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:25.908115    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"
Oct 29 23:30:26.070813 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.069812    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-kube-api-access-xxr2q\" (UniqueName: \"kubernetes.io/projected/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-kube-api-access-xxr2q\") pod \"23d0bc6c-7c9a-48aa-9722-ccb944f0a912\" (UID: \"23d0bc6c-7c9a-48aa-9722-ccb944f0a912\") "
Oct 29 23:30:26.070813 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.069885    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-maas-api-service-ca\") pod \"23d0bc6c-7c9a-48aa-9722-ccb944f0a912\" (UID: \"23d0bc6c-7c9a-48aa-9722-ccb944f0a912\") "
Oct 29 23:30:26.070813 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.070588    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-maas-api-service-ca" pod "23d0bc6c-7c9a-48aa-9722-ccb944f0a912" (UID: "23d0bc6c-7c9a-48aa-9722-ccb944f0a912"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:26.071946 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.071918    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-kube-api-access-xxr2q" pod "23d0bc6c-7c9a-48aa-9722-ccb944f0a912" (UID: "23d0bc6c-7c9a-48aa-9722-ccb944f0a912"). InnerVolumeSpecName "kube-api-access-xxr2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:26.174762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.174695    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxr2q\" (UniqueName: \"kubernetes.io/projected/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-kube-api-access-xxr2q\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:26.174966 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.174776    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23d0bc6c-7c9a-48aa-9722-ccb944f0a912-maas-api-service-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:30:26.372030 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.371937    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:26.372341 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.372320    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:30:26.372496 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.372473    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 29 23:30:26.409932 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.409521    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:26.414256 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.414227    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:30:26.414524 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.414488    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 29 23:30:26.439782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.438761    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8853a3df-1182-45aa-99e4-4272aef97e7b" path="/var/lib/kubelet/pods/8853a3df-1182-45aa-99e4-4272aef97e7b/volumes"
Oct 29 23:30:26.475513 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.475482    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"
Oct 29 23:30:26.476211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.476191    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r" event={"ID":"23d0bc6c-7c9a-48aa-9722-ccb944f0a912","Type":"ContainerDied","Data":"61363a3425001ac261d4082395c58f6c425302c7a85899d07276d811674909ab"}
Oct 29 23:30:26.476371 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.476360    6040 scope.go:122] "RemoveContainer" containerID="3f220cf983ca1ce4cf8eea0009ef55e72e024329c4a2f224d3309ace98339cd6"
Oct 29 23:30:26.593283 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.593245    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"]
Oct 29 23:30:26.666771 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.664428    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-2a53c9-29888610-tdk6r"]
Oct 29 23:30:26.907822 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.907783    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 29 23:30:26.908101 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:26.908070    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" podUID="f45f800d-3877-415d-b832-92956ed12f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 29 23:30:27.371191 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:27.371156    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:30:27.371477 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:27.371450    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 29 23:30:27.409099 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:27.409065    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:30:27.409540 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:27.409503    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 29 23:30:27.905763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:27.905203    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 29 23:30:27.905763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:27.905280    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" podUID="f45f800d-3877-415d-b832-92956ed12f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 29 23:30:28.372608 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:28.372112    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:30:28.372608 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:28.372173    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 29 23:30:28.412115 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:28.412079    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:30:28.412432 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:28.412392    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 29 23:30:28.491276 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:28.491234    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" path="/var/lib/kubelet/pods/23d0bc6c-7c9a-48aa-9722-ccb944f0a912/volumes"
Oct 29 23:30:28.777763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:28.777662    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:30:28.902180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:28.902144    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 29 23:30:28.902452 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:28.902423    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" podUID="f45f800d-3877-415d-b832-92956ed12f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 29 23:30:29.371554 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:29.371109    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:30:29.371554 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:29.371172    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 29 23:30:29.421762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:29.421230    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:30:29.421762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:29.421295    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 29 23:30:29.876992 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:30:29.876922    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:30:29.903768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:29.903327    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 29 23:30:29.903768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:29.903382    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" podUID="f45f800d-3877-415d-b832-92956ed12f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 29 23:30:30.373144 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:30.373108    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:30:30.373879 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:30.373843    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 29 23:30:30.414762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:30.414134    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:30:30.414762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:30.414194    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 29 23:30:30.901836 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:30.901797    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 29 23:30:30.902145 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:30.902099    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" podUID="f45f800d-3877-415d-b832-92956ed12f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 29 23:30:31.234822 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:31.233842    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 29 23:30:31.371085 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:31.371048    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:30:31.371372 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:31.371340    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 29 23:30:31.411324 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:31.411290    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:30:31.411922 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:31.411892    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 29 23:30:31.475848 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:31.474822    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:30:31.544940 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:31.544838    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:31.581942 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:31.581899    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:31.905764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:31.905221    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 29 23:30:31.905764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:31.905296    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" podUID="f45f800d-3877-415d-b832-92956ed12f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 29 23:30:32.459149 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:32.459115    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:32.461411 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:32.461391    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:32.467894 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:32.467872    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:30:32.599771 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:32.599711    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:32.600881 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:32.600859    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:32.601655 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:32.601639    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:30:33.750132 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:33.750098    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:33.750591 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:33.750575    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:33.751341 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:33.751319    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:30:54.025191 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.025154    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"]
Oct 29 23:30:54.025920 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.025895    6040 cpu_manager.go:414] "containerMap: removing container" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerName="cleanup"
Oct 29 23:30:54.025920 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.025919    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerName="cleanup"
Oct 29 23:30:54.026077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.025952    6040 cpu_manager.go:414] "containerMap: removing container" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerName="cleanup"
Oct 29 23:30:54.026077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.025958    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerName="cleanup"
Oct 29 23:30:54.026077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.026054    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerName="cleanup"
Oct 29 23:30:54.026077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.026071    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerName="cleanup"
Oct 29 23:30:54.033825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.033796    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.054404 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.054379    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"]
Oct 29 23:30:54.213034 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.213000    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.214269 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.214241    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.214620 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.214603    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.214795 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.214780    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/751a2a59-c2db-498e-8d07-eba400912132-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.214968 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.214952    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.215123 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.215110    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.215247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.215235    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chqt6\" (UniqueName: \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-kube-api-access-chqt6\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.215377 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.215364    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.216290 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.216272    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/751a2a59-c2db-498e-8d07-eba400912132-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.317957    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/751a2a59-c2db-498e-8d07-eba400912132-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318013    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318036    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318326 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318118    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318326 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318140    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/751a2a59-c2db-498e-8d07-eba400912132-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318326 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318181    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318326 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318220    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318326 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318252    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chqt6\" (UniqueName: \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-kube-api-access-chqt6\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318326 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318293    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318641 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318460    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318641 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318519    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318641 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318611    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318979 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318949    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.318979 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.318973    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/751a2a59-c2db-498e-8d07-eba400912132-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.320512 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.320491    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.320723 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.320706    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/751a2a59-c2db-498e-8d07-eba400912132-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.341198 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.341168    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.341327 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.341243    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chqt6\" (UniqueName: \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-kube-api-access-chqt6\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.347808 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.347788    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:54.668203 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.668178    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"]
Oct 29 23:30:54.669438 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:30:54.669414    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751a2a59_c2db_498e_8d07_eba400912132.slice/crio-4b96b7c2e31ba65837dafb9e6c2bfaa2fb71fb05bb079ae86ed35c9a9c41ac21 WatchSource:0}: Error finding container 4b96b7c2e31ba65837dafb9e6c2bfaa2fb71fb05bb079ae86ed35c9a9c41ac21: Status 404 returned error can't find the container with id 4b96b7c2e31ba65837dafb9e6c2bfaa2fb71fb05bb079ae86ed35c9a9c41ac21
Oct 29 23:30:54.671827 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.671793    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:54.671923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.671856    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:54.671923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.671886    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:30:54.711287 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:54.711262    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" event={"ID":"751a2a59-c2db-498e-8d07-eba400912132","Type":"ContainerStarted","Data":"4b96b7c2e31ba65837dafb9e6c2bfaa2fb71fb05bb079ae86ed35c9a9c41ac21"}
Oct 29 23:30:55.734009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:55.733974    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" event={"ID":"751a2a59-c2db-498e-8d07-eba400912132","Type":"ContainerStarted","Data":"94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa"}
Oct 29 23:30:55.813101 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:55.809720    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" podStartSLOduration=2.809698316 podStartE2EDuration="2.809698316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:55.79282142 +0000 UTC m=+1035.487930311" watchObservedRunningTime="2026-10-29 23:30:55.809698316 +0000 UTC m=+1035.504807197"
Oct 29 23:30:56.331322 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:56.331278    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:30:56.347964 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:56.347936    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:30:56.349522 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:56.349496    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 29 23:30:56.349639 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:56.349541    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" podUID="751a2a59-c2db-498e-8d07-eba400912132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 29 23:30:57.348386 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:57.348352    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 29 23:30:57.348859 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:57.348424    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" podUID="751a2a59-c2db-498e-8d07-eba400912132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 29 23:30:58.348428 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:58.348387    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 29 23:30:58.348867 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:58.348457    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" podUID="751a2a59-c2db-498e-8d07-eba400912132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 29 23:30:59.348359 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.348323    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 29 23:30:59.348550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.348386    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" podUID="751a2a59-c2db-498e-8d07-eba400912132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 29 23:30:59.482069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.482032    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"]
Oct 29 23:30:59.482542 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.482528    6040 cpu_manager.go:414] "containerMap: removing container" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerName="cleanup"
Oct 29 23:30:59.482611 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.482543    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerName="cleanup"
Oct 29 23:30:59.482676 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.482666    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="23d0bc6c-7c9a-48aa-9722-ccb944f0a912" containerName="cleanup"
Oct 29 23:30:59.486223 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.486201    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.533760 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.533051    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-53c44e\"/\"distinct-2-w5-53c44e-kserve-self-signed-certs\""
Oct 29 23:30:59.538224 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.538196    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"]
Oct 29 23:30:59.586806 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.586771    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-tmp-dir\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.587001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.586976    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-home\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.587059 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.587025    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rczp\" (UniqueName: \"kubernetes.io/projected/92ba9750-151b-4341-a4c4-7751e47dd8b5-kube-api-access-5rczp\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.587139 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.587123    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-dshm\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.587193 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.587153    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-model-cache\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.587193 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.587174    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ba9750-151b-4341-a4c4-7751e47dd8b5-tls-certs\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.688295 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.688255    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-dshm\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.689200 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.689173    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-model-cache\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.689362 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.689214    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ba9750-151b-4341-a4c4-7751e47dd8b5-tls-certs\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.689362 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.689301    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-tmp-dir\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.689478 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.689465    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-home\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.689533 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.689507    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rczp\" (UniqueName: \"kubernetes.io/projected/92ba9750-151b-4341-a4c4-7751e47dd8b5-kube-api-access-5rczp\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.690254 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.690225    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-tmp-dir\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.690401 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.690377    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-model-cache\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.690498 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.690477    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-home\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.697068 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.697012    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-dshm\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.698128 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.698104    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ba9750-151b-4341-a4c4-7751e47dd8b5-tls-certs\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.723514 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.723483    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rczp\" (UniqueName: \"kubernetes.io/projected/92ba9750-151b-4341-a4c4-7751e47dd8b5-kube-api-access-5rczp\") pod \"distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.811768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:30:59.807977    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:30:59.858940 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:30:59.858903    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:31:00.232270 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:31:00.232226    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92ba9750_151b_4341_a4c4_7751e47dd8b5.slice/crio-8da0a21407307cdd58bb229f375c822238da7b50de62495e14f0a92e58edbe81 WatchSource:0}: Error finding container 8da0a21407307cdd58bb229f375c822238da7b50de62495e14f0a92e58edbe81: Status 404 returned error can't find the container with id 8da0a21407307cdd58bb229f375c822238da7b50de62495e14f0a92e58edbe81
Oct 29 23:31:00.233467 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:00.233437    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"]
Oct 29 23:31:00.348987 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:00.348960    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 29 23:31:00.349352 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:00.349007    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" podUID="751a2a59-c2db-498e-8d07-eba400912132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 29 23:31:00.792198 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:00.792153    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" event={"ID":"92ba9750-151b-4341-a4c4-7751e47dd8b5","Type":"ContainerStarted","Data":"f4f9268fde1e7870e6206309f13fc4b6f890ace147ea0b4ef80f01f2cddc5bf6"}
Oct 29 23:31:00.792399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:00.792209    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" event={"ID":"92ba9750-151b-4341-a4c4-7751e47dd8b5","Type":"ContainerStarted","Data":"8da0a21407307cdd58bb229f375c822238da7b50de62495e14f0a92e58edbe81"}
Oct 29 23:31:00.839453 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:00.839405    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" podStartSLOduration=1.839390452 podStartE2EDuration="1.839390452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:00.837093582 +0000 UTC m=+1040.532202462" watchObservedRunningTime="2026-10-29 23:31:00.839390452 +0000 UTC m=+1040.534499383"
Oct 29 23:31:00.842151 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:00.842128    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:31:01.348143 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:01.348106    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 29 23:31:01.348316 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:01.348168    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" podUID="751a2a59-c2db-498e-8d07-eba400912132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 29 23:31:02.352846 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:02.352814    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:31:02.353216 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:02.353183    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:31:02.353851 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:02.353831    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:31:08.744977 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.744944    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"]
Oct 29 23:31:08.749352 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.749331    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.781407 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.781380    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"]
Oct 29 23:31:08.826432 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.826400    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.826432 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.826435    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.826682 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.826539    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94cqs\" (UniqueName: \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-kube-api-access-94cqs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.826682 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.826668    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.826826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.826704    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be781867-f314-44b4-8899-e66412a92d3d-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.826826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.826755    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.826966 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.826944    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be781867-f314-44b4-8899-e66412a92d3d-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.827024 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.826979    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.827024 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.827012    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.928331 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.928295    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.928331 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.928336    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be781867-f314-44b4-8899-e66412a92d3d-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.928550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.928356    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.928550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.928427    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be781867-f314-44b4-8899-e66412a92d3d-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.928550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.928458    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.928550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.928488    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.928803 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.928557    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.928803 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.928579    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.928803 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.928630    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94cqs\" (UniqueName: \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-kube-api-access-94cqs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.928980 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.928905    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.928980 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.928962    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.929079 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.929058    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.929134 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.929114    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.929314 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.929290    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be781867-f314-44b4-8899-e66412a92d3d-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.930497 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.930476    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.930790 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.930772    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be781867-f314-44b4-8899-e66412a92d3d-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.951355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.951333    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:08.951504 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:08.951485    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94cqs\" (UniqueName: \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-kube-api-access-94cqs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:09.066512 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:09.066436    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:09.386628 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:09.386592    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"]
Oct 29 23:31:09.396122 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:09.396083    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:09.396242 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:09.396169    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:09.396242 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:09.396217    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:09.841903 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:09.841818    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:31:09.850310 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:09.850275    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" event={"ID":"be781867-f314-44b4-8899-e66412a92d3d","Type":"ContainerStarted","Data":"463f241c32fdfa045a165e30e11ac524ac21c6bf4b5cc2946b608c1f3ff77a59"}
Oct 29 23:31:09.850480 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:09.850333    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" event={"ID":"be781867-f314-44b4-8899-e66412a92d3d","Type":"ContainerStarted","Data":"e6c8b07e70a4eae250d3b6129020c45df3de47bfcc162743289f69a7f9106a46"}
Oct 29 23:31:09.855539 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:09.855518    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:31:09.867039 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:09.867015    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:31:09.897987 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:09.897174    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" podStartSLOduration=1.8971550000000001 podStartE2EDuration="1.897155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:09.895530568 +0000 UTC m=+1049.590639448" watchObservedRunningTime="2026-10-29 23:31:09.897155 +0000 UTC m=+1049.592263883"
Oct 29 23:31:10.066621 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:10.066588    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:10.068293 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:10.068265    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 29 23:31:10.068418 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:10.068312    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 29 23:31:11.067116 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.067085    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 29 23:31:11.067511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.067144    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 29 23:31:11.635947 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.635913    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"]
Oct 29 23:31:11.640028 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.640005    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.666695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.666665    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b978a0\"/\"distinct-2-w0-b978a0-kserve-self-signed-certs\""
Oct 29 23:31:11.670444 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.670417    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"]
Oct 29 23:31:11.762509 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.762475    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tmp-dir\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.762707 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.762541    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ts8d\" (UniqueName: \"kubernetes.io/projected/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-kube-api-access-4ts8d\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.762707 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.762686    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-model-cache\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.762821 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.762720    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-dshm\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.762968 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.762944    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-home\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.763033 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.762984    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tls-certs\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.866569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.864057    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-home\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.866569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.864108    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tls-certs\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.866569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.864147    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tmp-dir\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.866569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.864223    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ts8d\" (UniqueName: \"kubernetes.io/projected/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-kube-api-access-4ts8d\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.866569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.864328    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-model-cache\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.866569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.864377    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-dshm\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.880807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.872014    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tmp-dir\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.880807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.872264    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-home\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.880807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.878339    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-dshm\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.880807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.878979    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-model-cache\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.880807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.879344    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tls-certs\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.919908 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.914659    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ts8d\" (UniqueName: \"kubernetes.io/projected/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-kube-api-access-4ts8d\") pod \"distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") " pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:11.955879 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:11.955842    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:12.067729 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:12.067467    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 29 23:31:12.067729 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:12.067537    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 29 23:31:12.331716 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:12.331594    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"]
Oct 29 23:31:12.335921 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:31:12.335885    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0bd477c5_2f9a_46c8_a6d6_078f2ec7bbf0.slice/crio-1a8c9078e9b822c4b5f6160d0c72a4f96b8bf59b48c00177c49b11cf1b2c85fa WatchSource:0}: Error finding container 1a8c9078e9b822c4b5f6160d0c72a4f96b8bf59b48c00177c49b11cf1b2c85fa: Status 404 returned error can't find the container with id 1a8c9078e9b822c4b5f6160d0c72a4f96b8bf59b48c00177c49b11cf1b2c85fa
Oct 29 23:31:12.888646 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:12.888613    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" event={"ID":"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0","Type":"ContainerStarted","Data":"b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73"}
Oct 29 23:31:12.888835 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:12.888654    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" event={"ID":"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0","Type":"ContainerStarted","Data":"1a8c9078e9b822c4b5f6160d0c72a4f96b8bf59b48c00177c49b11cf1b2c85fa"}
Oct 29 23:31:12.956949 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:12.956893    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podStartSLOduration=1.956876336 podStartE2EDuration="1.956876336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:12.954020844 +0000 UTC m=+1052.649129747" watchObservedRunningTime="2026-10-29 23:31:12.956876336 +0000 UTC m=+1052.651985272"
Oct 29 23:31:12.957141 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:12.957055    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:13.067129 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:13.067091    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 29 23:31:13.067320 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:13.067157    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 29 23:31:14.067357 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:14.067319    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 29 23:31:14.067730 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:14.067387    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 29 23:31:15.067278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:15.067244    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 29 23:31:15.067470 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:15.067300    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 29 23:31:16.067717 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:16.067680    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 29 23:31:16.068088 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:16.067732    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 29 23:31:17.128761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:17.128074    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:17.132759 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:17.129579    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:17.140759 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:17.138984    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:31:21.956808 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:21.956774    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:21.969193 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:21.969171    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:21.992680 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:21.992654    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:31:23.779795 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.779762    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"]
Oct 29 23:31:23.783449 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.783430    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.809058 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.809036    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"]
Oct 29 23:31:23.868652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.868609    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.868827 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.868671    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8pvn\" (UniqueName: \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-kube-api-access-w8pvn\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.868827 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.868781    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.868950 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.868830    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7691e1-6b13-4179-a5f5-5a7af04c6525-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.868950 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.868887    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.868950 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.868917    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.869067 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.868989    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.869105 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.869061    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.869171 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.869157    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.970588 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.970558    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.970794 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.970623    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.970794 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.970652    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8pvn\" (UniqueName: \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-kube-api-access-w8pvn\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.970794 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.970696    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.970794 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.970776    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7691e1-6b13-4179-a5f5-5a7af04c6525-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.971035 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.970803    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.971035 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.970828    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.971035 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.970884    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.971035 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.970953    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.971230 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.971148    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.971230 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.971208    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.971334 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.971286    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.971388 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.971337    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.971541 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.971522    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7691e1-6b13-4179-a5f5-5a7af04c6525-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.973277 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.973240    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.973462 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.973443    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.997019 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.996991    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:23.997356 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:23.997334    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8pvn\" (UniqueName: \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-kube-api-access-w8pvn\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:24.097301 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:24.097224    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:24.426888 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:24.426822    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"]
Oct 29 23:31:24.429287 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:31:24.429260    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe7691e1_6b13_4179_a5f5_5a7af04c6525.slice/crio-acda1dfa91ff0ab65db624ce7bea7e7812c0119148cc74c73d1e1423c295e5dc WatchSource:0}: Error finding container acda1dfa91ff0ab65db624ce7bea7e7812c0119148cc74c73d1e1423c295e5dc: Status 404 returned error can't find the container with id acda1dfa91ff0ab65db624ce7bea7e7812c0119148cc74c73d1e1423c295e5dc
Oct 29 23:31:24.431667 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:24.431639    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:24.431721 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:24.431706    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:24.431791 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:24.431764    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:25.010897 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:25.010855    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" event={"ID":"be7691e1-6b13-4179-a5f5-5a7af04c6525","Type":"ContainerStarted","Data":"5372e7a0d1fa99c442c2f2bfc7b792b459272039737483cab609051f3280aa40"}
Oct 29 23:31:25.011389 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:25.010922    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" event={"ID":"be7691e1-6b13-4179-a5f5-5a7af04c6525","Type":"ContainerStarted","Data":"acda1dfa91ff0ab65db624ce7bea7e7812c0119148cc74c73d1e1423c295e5dc"}
Oct 29 23:31:25.083708 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:25.083656    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" podStartSLOduration=2.083642496 podStartE2EDuration="2.083642496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:25.069022239 +0000 UTC m=+1064.764131118" watchObservedRunningTime="2026-10-29 23:31:25.083642496 +0000 UTC m=+1064.778751374"
Oct 29 23:31:25.098173 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:25.098141    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:25.099806 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:25.099785    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 29 23:31:25.099908 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:25.099825    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 29 23:31:26.098259 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:26.098223    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 29 23:31:26.098691 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:26.098285    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 29 23:31:26.995615 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:26.995576    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:31:27.098556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:27.098523    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 29 23:31:27.098976 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:27.098571    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 29 23:31:28.098406 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:28.098371    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 29 23:31:28.098579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:28.098436    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 29 23:31:29.097920 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:29.097878    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 29 23:31:29.098133 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:29.097945    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 29 23:31:29.782440 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:31:29.782396    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:31:30.098049 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:30.097968    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 29 23:31:30.098049 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:30.098028    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 29 23:31:31.097637 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:31.097596    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 29 23:31:31.098138 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:31.097664    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 29 23:31:32.102340 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:32.102311    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:32.102831 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:32.102767    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:32.103382 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:32.103364    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:31:47.370073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:47.370028    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:47.485762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:47.485605    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:47.510811 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:47.504950    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:31:47.593633 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:47.593585    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:47.653769 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:47.652865    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:47.865883 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:47.865835    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:47.985993 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:47.985883    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 29 23:31:48.137477 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.137425    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:31:48.137815 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.137785    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:48.179905 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.179867    6040 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:31:48.180200 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.180155    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:48.252727 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.252624    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": context deadline exceeded"
Oct 29 23:31:48.253439 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.253401    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 29 23:31:48.292936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.282647    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:48.379996 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.379944    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:48.381103 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.381081    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:31:48.418782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.408131    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:48.424782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.419080    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:31:48.424782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.419135    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" podUID="751a2a59-c2db-498e-8d07-eba400912132" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:48.486956 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.486829    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:31:48.490814 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.490790    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:31:48.524958 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.524859    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:31:48.525294 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.525267    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:31:48.601762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.600816    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 29 23:31:48.601762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.601218    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:31:48.788439 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.788357    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:31:48.819773 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.818286    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:31:48.901683 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.880012    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:31:48.915626 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:48.915596    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:31:53.060407 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.060328    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"]
Oct 29 23:31:53.065503 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.065486    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:53.083421 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.083397    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"]
Oct 29 23:31:53.084489 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.084462    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0e30e0cc-tfd5c\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:53.084753 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.084714    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb2ns\" (UniqueName: \"kubernetes.io/projected/e6e92817-391d-4736-ab23-0de8643a7bbc-kube-api-access-fb2ns\") pod \"maas-api-revoke-sub-0e30e0cc-tfd5c\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:53.085004 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.084979    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0e30e0cc-tfd5c\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:53.091255 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.091235    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:31:53.091354 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.091272    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:31:53.186109 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.186074    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0e30e0cc-tfd5c\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:53.186289 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.186137    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0e30e0cc-tfd5c\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:53.186289 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.186220    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb2ns\" (UniqueName: \"kubernetes.io/projected/e6e92817-391d-4736-ab23-0de8643a7bbc-kube-api-access-fb2ns\") pod \"maas-api-revoke-sub-0e30e0cc-tfd5c\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:53.186754 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.186724    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0e30e0cc-tfd5c\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:53.188570 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.188553    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0e30e0cc-tfd5c\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:53.209414 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.209386    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb2ns\" (UniqueName: \"kubernetes.io/projected/e6e92817-391d-4736-ab23-0de8643a7bbc-kube-api-access-fb2ns\") pod \"maas-api-revoke-sub-0e30e0cc-tfd5c\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:53.378368 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.378290    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:53.691329 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.691296    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"]
Oct 29 23:31:53.692973 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:31:53.692947    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e92817_391d_4736_ab23_0de8643a7bbc.slice/crio-4342a95ffeeb6525a9aa12e563a509f5ab2ebcfb8f4ffa14e989bad9d3ce88bf WatchSource:0}: Error finding container 4342a95ffeeb6525a9aa12e563a509f5ab2ebcfb8f4ffa14e989bad9d3ce88bf: Status 404 returned error can't find the container with id 4342a95ffeeb6525a9aa12e563a509f5ab2ebcfb8f4ffa14e989bad9d3ce88bf
Oct 29 23:31:53.774417 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.774382    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"]
Oct 29 23:31:53.779407 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.779375    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.805105 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.805063    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"]
Oct 29 23:31:53.813087 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.813057    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"]
Oct 29 23:31:53.818076 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.818052    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.852635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.844286    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"]
Oct 29 23:31:53.894810 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.894778    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd7b03f-a363-415c-9397-5d202774916d-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.894982 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.894846    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.894982 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.894889    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f5qg\" (UniqueName: \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-kube-api-access-6f5qg\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.895135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895105    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.895201 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895148    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.895201 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895174    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.895306 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895203    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.895306 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895238    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.895306 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895266    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.895306 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895290    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6df2\" (UniqueName: \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-kube-api-access-v6df2\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.895491 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895324    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd7b03f-a363-415c-9397-5d202774916d-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.895491 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895356    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.895491 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895416    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.895635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895495    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.895678 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895659    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.895780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895731    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.895861 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895846    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.895906 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.895874    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.997313 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997227    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.997313 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997284    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f5qg\" (UniqueName: \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-kube-api-access-6f5qg\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.997549 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997343    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.997549 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997368    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.997549 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997394    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.997549 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997423    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.997549 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997463    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.997849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997587    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.997849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997631    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6df2\" (UniqueName: \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-kube-api-access-v6df2\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.997849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997770    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd7b03f-a363-415c-9397-5d202774916d-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.997849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997817    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.998057 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997880    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.998057 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997922    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.998057 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.997968    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.998207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.998067    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.998207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.998088    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.998207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.998183    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:53.998364 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.998210    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.998364 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.998258    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd7b03f-a363-415c-9397-5d202774916d-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.998443 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.998422    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.998724 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.998692    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.999022 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.998997    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:53.999127 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.999019    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd7b03f-a363-415c-9397-5d202774916d-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:54.000118 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.999341    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:54.000118 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.999492    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:54.000118 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.999534    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:54.000118 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:53.999686    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:54.000118 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.000044    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:54.001964 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.001938    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:54.003243 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.003215    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:54.003686 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.003663    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:54.005163 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.005138    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd7b03f-a363-415c-9397-5d202774916d-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:54.021256 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.021201    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f5qg\" (UniqueName: \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-kube-api-access-6f5qg\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:54.022573 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.022545    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:54.022771 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.022732    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:54.023267 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.023244    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6df2\" (UniqueName: \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-kube-api-access-v6df2\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:54.093497 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.093461    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:54.139296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.139254    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:54.396407 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.396373    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c" event={"ID":"e6e92817-391d-4736-ab23-0de8643a7bbc","Type":"ContainerStarted","Data":"fd200a663770d07bce988005c29979bd42acdb114ff03e65320d053200b3a166"}
Oct 29 23:31:54.396563 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.396435    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c" event={"ID":"e6e92817-391d-4736-ab23-0de8643a7bbc","Type":"ContainerStarted","Data":"4342a95ffeeb6525a9aa12e563a509f5ab2ebcfb8f4ffa14e989bad9d3ce88bf"}
Oct 29 23:31:54.415829 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.415803    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"]
Oct 29 23:31:54.417499 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:31:54.417467    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd7b03f_a363_415c_9397_5d202774916d.slice/crio-c519b56bb3632f202a6dd5c1133f8c4523b1a321a4667264e7c521a5ce2a9181 WatchSource:0}: Error finding container c519b56bb3632f202a6dd5c1133f8c4523b1a321a4667264e7c521a5ce2a9181: Status 404 returned error can't find the container with id c519b56bb3632f202a6dd5c1133f8c4523b1a321a4667264e7c521a5ce2a9181
Oct 29 23:31:54.420804 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.420761    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:54.420910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.420850    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:54.420910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.420895    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:54.455531 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.455472    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c" podStartSLOduration=1.455453933 podStartE2EDuration="1.455453933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:54.443930611 +0000 UTC m=+1094.139039491" watchObservedRunningTime="2026-10-29 23:31:54.455453933 +0000 UTC m=+1094.150562812"
Oct 29 23:31:54.458293 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.458239    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"]
Oct 29 23:31:54.460978 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:31:54.460945    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d13594_12eb_4f2c_8ce7_6c92cfec635d.slice/crio-5510e21f2d1227adc856211f837f78be809b3e1e59f454ba95937e0685ffe4ad WatchSource:0}: Error finding container 5510e21f2d1227adc856211f837f78be809b3e1e59f454ba95937e0685ffe4ad: Status 404 returned error can't find the container with id 5510e21f2d1227adc856211f837f78be809b3e1e59f454ba95937e0685ffe4ad
Oct 29 23:31:54.463675 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.463644    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:54.463788 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.463707    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:54.463788 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:54.463751    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:31:55.451472 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:55.451377    6040 generic.go:356] "Generic (PLEG): container finished" podID="e6e92817-391d-4736-ab23-0de8643a7bbc" containerID="fd200a663770d07bce988005c29979bd42acdb114ff03e65320d053200b3a166" exitCode=0
Oct 29 23:31:55.452197 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:55.452168    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c" event={"ID":"e6e92817-391d-4736-ab23-0de8643a7bbc","Type":"ContainerDied","Data":"fd200a663770d07bce988005c29979bd42acdb114ff03e65320d053200b3a166"}
Oct 29 23:31:55.455146 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:55.455119    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" event={"ID":"42d13594-12eb-4f2c-8ce7-6c92cfec635d","Type":"ContainerStarted","Data":"e8e95ceaff603a3321ed7649b720b89f79c599b719da41e6af30812bcf23367c"}
Oct 29 23:31:55.455268 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:55.455170    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" event={"ID":"42d13594-12eb-4f2c-8ce7-6c92cfec635d","Type":"ContainerStarted","Data":"5510e21f2d1227adc856211f837f78be809b3e1e59f454ba95937e0685ffe4ad"}
Oct 29 23:31:55.457614 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:55.457590    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" event={"ID":"9fd7b03f-a363-415c-9397-5d202774916d","Type":"ContainerStarted","Data":"4daa2be0fbcf31fb559427af6f300a038f84b66e5c8d3d5f4a19091c6ae4dafd"}
Oct 29 23:31:55.457733 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:55.457638    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" event={"ID":"9fd7b03f-a363-415c-9397-5d202774916d","Type":"ContainerStarted","Data":"c519b56bb3632f202a6dd5c1133f8c4523b1a321a4667264e7c521a5ce2a9181"}
Oct 29 23:31:55.555574 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:55.555511    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" podStartSLOduration=2.555493178 podStartE2EDuration="2.555493178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:55.555093716 +0000 UTC m=+1095.250202596" watchObservedRunningTime="2026-10-29 23:31:55.555493178 +0000 UTC m=+1095.250602057"
Oct 29 23:31:56.093858 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.093822    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:31:56.095168 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.095145    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:31:56.095262 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.095195    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" podUID="9fd7b03f-a363-415c-9397-5d202774916d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 29 23:31:56.142762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.142100    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:31:56.144540 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.144508    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:31:56.144791 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.144764    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" podUID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 29 23:31:56.775535 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.775508    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:56.855762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.853403    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" podStartSLOduration=3.853382747 podStartE2EDuration="3.853382747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:55.611988717 +0000 UTC m=+1095.307097597" watchObservedRunningTime="2026-10-29 23:31:56.853382747 +0000 UTC m=+1096.548491628"
Oct 29 23:31:56.872474 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.872439    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6e92817-391d-4736-ab23-0de8643a7bbc-kube-api-access-fb2ns\" (UniqueName: \"kubernetes.io/projected/e6e92817-391d-4736-ab23-0de8643a7bbc-kube-api-access-fb2ns\") pod \"e6e92817-391d-4736-ab23-0de8643a7bbc\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") "
Oct 29 23:31:56.872641 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.872528    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-openshift-ca\") pod \"e6e92817-391d-4736-ab23-0de8643a7bbc\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") "
Oct 29 23:31:56.872641 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.872597    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-xks-ca\") pod \"e6e92817-391d-4736-ab23-0de8643a7bbc\" (UID: \"e6e92817-391d-4736-ab23-0de8643a7bbc\") "
Oct 29 23:31:56.873228 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.873193    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-openshift-ca" pod "e6e92817-391d-4736-ab23-0de8643a7bbc" (UID: "e6e92817-391d-4736-ab23-0de8643a7bbc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:56.875103 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.875075    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6e92817-391d-4736-ab23-0de8643a7bbc-kube-api-access-fb2ns" pod "e6e92817-391d-4736-ab23-0de8643a7bbc" (UID: "e6e92817-391d-4736-ab23-0de8643a7bbc"). InnerVolumeSpecName "kube-api-access-fb2ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:56.875469 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.875448    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-xks-ca" pod "e6e92817-391d-4736-ab23-0de8643a7bbc" (UID: "e6e92817-391d-4736-ab23-0de8643a7bbc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:31:56.974693 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.974660    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb2ns\" (UniqueName: \"kubernetes.io/projected/e6e92817-391d-4736-ab23-0de8643a7bbc-kube-api-access-fb2ns\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:31:56.974693 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.974691    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:31:56.974693 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:56.974702    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6e92817-391d-4736-ab23-0de8643a7bbc-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:31:57.094318 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:57.094234    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:31:57.094318 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:57.094303    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" podUID="9fd7b03f-a363-415c-9397-5d202774916d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 29 23:31:57.150763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:57.150631    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:31:57.150763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:57.150704    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" podUID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 29 23:31:57.484479 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:57.484445    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"
Oct 29 23:31:57.484652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:57.484482    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c" event={"ID":"e6e92817-391d-4736-ab23-0de8643a7bbc","Type":"ContainerDied","Data":"4342a95ffeeb6525a9aa12e563a509f5ab2ebcfb8f4ffa14e989bad9d3ce88bf"}
Oct 29 23:31:57.484652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:57.484527    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4342a95ffeeb6525a9aa12e563a509f5ab2ebcfb8f4ffa14e989bad9d3ce88bf"
Oct 29 23:31:58.094512 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:58.094479    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:31:58.094972 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:58.094538    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" podUID="9fd7b03f-a363-415c-9397-5d202774916d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 29 23:31:58.142761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:58.142141    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:31:58.142761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:58.142203    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" podUID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 29 23:31:59.094118 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:59.094084    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:31:59.094309 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:59.094139    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" podUID="9fd7b03f-a363-415c-9397-5d202774916d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 29 23:31:59.140760 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:59.140192    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:31:59.140760 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:31:59.140258    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" podUID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 29 23:31:59.813256 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:31:59.813201    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:32:00.094804 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:00.094712    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:32:00.095017 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:00.094994    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" podUID="9fd7b03f-a363-415c-9397-5d202774916d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 29 23:32:00.143259 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:00.143221    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:32:00.143900 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:00.143854    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" podUID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 29 23:32:01.094879 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:01.094819    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:32:01.095073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:01.094904    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" podUID="9fd7b03f-a363-415c-9397-5d202774916d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 29 23:32:01.140825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:01.140780    6040 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:32:01.141038 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:01.140901    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" podUID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 29 23:32:02.153773 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:02.153706    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:32:02.155023 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:02.154997    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:32:02.156066 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:02.155997    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:32:02.199598 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:02.199567    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:32:02.200040 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:02.200017    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:32:02.201170 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:02.201146    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:32:03.385597 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.385564    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"]
Oct 29 23:32:03.386192 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.386173    6040 cpu_manager.go:414] "containerMap: removing container" podUID="e6e92817-391d-4736-ab23-0de8643a7bbc" containerName="revoke-keys"
Oct 29 23:32:03.386264 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.386194    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6e92817-391d-4736-ab23-0de8643a7bbc" containerName="revoke-keys"
Oct 29 23:32:03.386367 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.386356    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6e92817-391d-4736-ab23-0de8643a7bbc" containerName="revoke-keys"
Oct 29 23:32:03.389912 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.389896    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:03.408508 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.408484    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"]
Oct 29 23:32:03.411767 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.411717    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:32:03.412127 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.412105    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:32:03.435126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.435096    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d4f8c666-l499q\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:03.435285 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.435149    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d4f8c666-l499q\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:03.435436 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.435401    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g62sb\" (UniqueName: \"kubernetes.io/projected/ac38c214-7803-4b30-ba48-76dd80495c11-kube-api-access-g62sb\") pod \"maas-api-revoke-sub-d4f8c666-l499q\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:03.536796 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.536764    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d4f8c666-l499q\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:03.536972 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.536808    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d4f8c666-l499q\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:03.536972 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.536875    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g62sb\" (UniqueName: \"kubernetes.io/projected/ac38c214-7803-4b30-ba48-76dd80495c11-kube-api-access-g62sb\") pod \"maas-api-revoke-sub-d4f8c666-l499q\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:03.537534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.537508    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d4f8c666-l499q\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:03.539218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.539193    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d4f8c666-l499q\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:03.563017 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.562990    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g62sb\" (UniqueName: \"kubernetes.io/projected/ac38c214-7803-4b30-ba48-76dd80495c11-kube-api-access-g62sb\") pod \"maas-api-revoke-sub-d4f8c666-l499q\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:03.704312 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:03.704233    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:04.006794 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:04.006765    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"]
Oct 29 23:32:04.007341 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:32:04.007307    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac38c214_7803_4b30_ba48_76dd80495c11.slice/crio-21b1eb1b90f54612b60d76c731d15c6837c6a5f5b67181860eb068164aa3a53d WatchSource:0}: Error finding container 21b1eb1b90f54612b60d76c731d15c6837c6a5f5b67181860eb068164aa3a53d: Status 404 returned error can't find the container with id 21b1eb1b90f54612b60d76c731d15c6837c6a5f5b67181860eb068164aa3a53d
Oct 29 23:32:04.557531 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:04.557499    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q" event={"ID":"ac38c214-7803-4b30-ba48-76dd80495c11","Type":"ContainerStarted","Data":"14a47f68187835a4bccb7588bf9c37c755d08c9af459445ff969713197cc4c1e"}
Oct 29 23:32:04.558009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:04.557558    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q" event={"ID":"ac38c214-7803-4b30-ba48-76dd80495c11","Type":"ContainerStarted","Data":"21b1eb1b90f54612b60d76c731d15c6837c6a5f5b67181860eb068164aa3a53d"}
Oct 29 23:32:04.599642 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:04.599543    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q" podStartSLOduration=1.599526792 podStartE2EDuration="1.599526792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:32:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:32:04.597229379 +0000 UTC m=+1104.292338259" watchObservedRunningTime="2026-10-29 23:32:04.599526792 +0000 UTC m=+1104.294635670"
Oct 29 23:32:05.565346 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:05.565310    6040 generic.go:356] "Generic (PLEG): container finished" podID="ac38c214-7803-4b30-ba48-76dd80495c11" containerID="14a47f68187835a4bccb7588bf9c37c755d08c9af459445ff969713197cc4c1e" exitCode=0
Oct 29 23:32:05.565767 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:05.565397    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q" event={"ID":"ac38c214-7803-4b30-ba48-76dd80495c11","Type":"ContainerDied","Data":"14a47f68187835a4bccb7588bf9c37c755d08c9af459445ff969713197cc4c1e"}
Oct 29 23:32:06.717595 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:06.717569    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:06.788765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:06.788221    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-openshift-ca\") pod \"ac38c214-7803-4b30-ba48-76dd80495c11\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") "
Oct 29 23:32:06.788765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:06.788347    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac38c214-7803-4b30-ba48-76dd80495c11-kube-api-access-g62sb\" (UniqueName: \"kubernetes.io/projected/ac38c214-7803-4b30-ba48-76dd80495c11-kube-api-access-g62sb\") pod \"ac38c214-7803-4b30-ba48-76dd80495c11\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") "
Oct 29 23:32:06.788765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:06.788481    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-xks-ca\") pod \"ac38c214-7803-4b30-ba48-76dd80495c11\" (UID: \"ac38c214-7803-4b30-ba48-76dd80495c11\") "
Oct 29 23:32:06.789600 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:06.789546    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-openshift-ca" pod "ac38c214-7803-4b30-ba48-76dd80495c11" (UID: "ac38c214-7803-4b30-ba48-76dd80495c11"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:32:06.792238 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:06.792205    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-xks-ca" pod "ac38c214-7803-4b30-ba48-76dd80495c11" (UID: "ac38c214-7803-4b30-ba48-76dd80495c11"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:32:06.792368 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:06.792251    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac38c214-7803-4b30-ba48-76dd80495c11-kube-api-access-g62sb" pod "ac38c214-7803-4b30-ba48-76dd80495c11" (UID: "ac38c214-7803-4b30-ba48-76dd80495c11"). InnerVolumeSpecName "kube-api-access-g62sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:32:06.891566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:06.891537    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:06.891566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:06.891560    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g62sb\" (UniqueName: \"kubernetes.io/projected/ac38c214-7803-4b30-ba48-76dd80495c11-kube-api-access-g62sb\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:06.891566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:06.891574    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac38c214-7803-4b30-ba48-76dd80495c11-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:07.579140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:07.579111    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"
Oct 29 23:32:07.579307 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:07.579112    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q" event={"ID":"ac38c214-7803-4b30-ba48-76dd80495c11","Type":"ContainerDied","Data":"21b1eb1b90f54612b60d76c731d15c6837c6a5f5b67181860eb068164aa3a53d"}
Oct 29 23:32:07.579307 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:07.579229    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21b1eb1b90f54612b60d76c731d15c6837c6a5f5b67181860eb068164aa3a53d"
Oct 29 23:32:07.850019 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:07.849940    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:32:10.220761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.219829    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:10.243115 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.241019    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 29 23:32:10.392762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.391841    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:10.543819 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.542832    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:10.642887 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.642830    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:10.795991 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.793367    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:32:10.826970 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.826909    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:10.851125 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.850810    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:10.864227 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.864187    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:10.888619 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.888196    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:32:10.975970 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.975923    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:10.983430 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:10.983352    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:11.222773 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:11.221204    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:11.222773 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:11.221997    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:32:11.242609 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:11.241275    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:11.242609 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:11.241716    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:32:11.262770 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:11.260801    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:32:11.289770 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:11.286295    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:32:11.398759 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:11.397908    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:11.398759 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:11.398682    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:32:12.079764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:12.076810    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": context deadline exceeded"
Oct 29 23:32:12.079764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:12.077120    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:32:12.079764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:12.078927    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:32:12.173764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:12.092205    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:12.173764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:12.105026    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:32:12.295786 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:12.282261    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:12.402488 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:12.402440    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:12.676987 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:12.645967    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:13.054119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.054015    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:13.064600 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.064555    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:13.074117 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.069785    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:13.093577 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.091165    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:13.095255 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.092410    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:32:13.123057 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.094222    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:13.123343 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.083190    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 29 23:32:13.291779 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.284817    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:13.314821 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.301835    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 29 23:32:13.350763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.348950    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:32:13.411762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.409868    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:13.683762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.659618    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:32:13.724764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.685253    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 29 23:32:13.728956 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.728931    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:32:13.756087 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.756057    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:32:13.832535 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.832361    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:32:13.864596 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.864440    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:32:13.893680 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:13.893653    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:32:18.940853 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:18.940818    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"]
Oct 29 23:32:18.941584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:18.941567    6040 cpu_manager.go:414] "containerMap: removing container" podUID="ac38c214-7803-4b30-ba48-76dd80495c11" containerName="revoke-keys"
Oct 29 23:32:18.941686 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:18.941585    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac38c214-7803-4b30-ba48-76dd80495c11" containerName="revoke-keys"
Oct 29 23:32:18.941760 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:18.941725    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac38c214-7803-4b30-ba48-76dd80495c11" containerName="revoke-keys"
Oct 29 23:32:18.945810 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:18.945791    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:18.964356 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:18.964326    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"]
Oct 29 23:32:18.971648 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:18.971385    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:32:18.971648 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:18.971486    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:32:19.004244 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:19.004207    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pbj6\" (UniqueName: \"kubernetes.io/projected/3a7a0d36-7039-4782-9835-c159ac1e836e-kube-api-access-6pbj6\") pod \"maas-api-revoke-sub-46c8d7a9-nfzpj\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:19.004442 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:19.004406    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-46c8d7a9-nfzpj\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:19.004666 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:19.004641    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-46c8d7a9-nfzpj\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:19.105934 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:19.105899    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-46c8d7a9-nfzpj\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:19.106120 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:19.106079    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-46c8d7a9-nfzpj\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:19.106169 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:19.106141    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pbj6\" (UniqueName: \"kubernetes.io/projected/3a7a0d36-7039-4782-9835-c159ac1e836e-kube-api-access-6pbj6\") pod \"maas-api-revoke-sub-46c8d7a9-nfzpj\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:19.106984 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:19.106959    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-46c8d7a9-nfzpj\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:19.108486 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:19.108467    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-46c8d7a9-nfzpj\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:19.130174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:19.130151    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pbj6\" (UniqueName: \"kubernetes.io/projected/3a7a0d36-7039-4782-9835-c159ac1e836e-kube-api-access-6pbj6\") pod \"maas-api-revoke-sub-46c8d7a9-nfzpj\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:19.258889 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:19.258798    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:19.570388 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:19.570304    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"]
Oct 29 23:32:19.575387 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:32:19.575350    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a7a0d36_7039_4782_9835_c159ac1e836e.slice/crio-ef5db7d350e694a8a11b19a1a0f2459c1b810abab82c774fc0a4661065380fc0 WatchSource:0}: Error finding container ef5db7d350e694a8a11b19a1a0f2459c1b810abab82c774fc0a4661065380fc0: Status 404 returned error can't find the container with id ef5db7d350e694a8a11b19a1a0f2459c1b810abab82c774fc0a4661065380fc0
Oct 29 23:32:20.242271 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:20.242173    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj" event={"ID":"3a7a0d36-7039-4782-9835-c159ac1e836e","Type":"ContainerStarted","Data":"91ffb3f54ef7003b9a11f944e32caffe78177f7261ee56d1afd41288ac0f618f"}
Oct 29 23:32:20.242271 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:20.242237    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj" event={"ID":"3a7a0d36-7039-4782-9835-c159ac1e836e","Type":"ContainerStarted","Data":"ef5db7d350e694a8a11b19a1a0f2459c1b810abab82c774fc0a4661065380fc0"}
Oct 29 23:32:20.282885 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:20.282818    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj" podStartSLOduration=2.282802962 podStartE2EDuration="2.282802962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:32:20.282015566 +0000 UTC m=+1119.977124446" watchObservedRunningTime="2026-10-29 23:32:20.282802962 +0000 UTC m=+1119.977911841"
Oct 29 23:32:21.250078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:21.250039    6040 generic.go:356] "Generic (PLEG): container finished" podID="3a7a0d36-7039-4782-9835-c159ac1e836e" containerID="91ffb3f54ef7003b9a11f944e32caffe78177f7261ee56d1afd41288ac0f618f" exitCode=0
Oct 29 23:32:21.250477 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:21.250125    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj" event={"ID":"3a7a0d36-7039-4782-9835-c159ac1e836e","Type":"ContainerDied","Data":"91ffb3f54ef7003b9a11f944e32caffe78177f7261ee56d1afd41288ac0f618f"}
Oct 29 23:32:22.413118 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:22.413088    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:22.450327 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:22.450297    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-xks-ca\") pod \"3a7a0d36-7039-4782-9835-c159ac1e836e\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") "
Oct 29 23:32:22.450500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:22.450333    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-openshift-ca\") pod \"3a7a0d36-7039-4782-9835-c159ac1e836e\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") "
Oct 29 23:32:22.450500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:22.450369    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a7a0d36-7039-4782-9835-c159ac1e836e-kube-api-access-6pbj6\" (UniqueName: \"kubernetes.io/projected/3a7a0d36-7039-4782-9835-c159ac1e836e-kube-api-access-6pbj6\") pod \"3a7a0d36-7039-4782-9835-c159ac1e836e\" (UID: \"3a7a0d36-7039-4782-9835-c159ac1e836e\") "
Oct 29 23:32:22.450784 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:22.450734    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-openshift-ca" pod "3a7a0d36-7039-4782-9835-c159ac1e836e" (UID: "3a7a0d36-7039-4782-9835-c159ac1e836e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:32:22.451235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:22.451082    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:22.452431 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:22.452408    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-xks-ca" pod "3a7a0d36-7039-4782-9835-c159ac1e836e" (UID: "3a7a0d36-7039-4782-9835-c159ac1e836e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:32:22.452505 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:22.452478    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a7a0d36-7039-4782-9835-c159ac1e836e-kube-api-access-6pbj6" pod "3a7a0d36-7039-4782-9835-c159ac1e836e" (UID: "3a7a0d36-7039-4782-9835-c159ac1e836e"). InnerVolumeSpecName "kube-api-access-6pbj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:32:22.552959 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:22.552881    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a7a0d36-7039-4782-9835-c159ac1e836e-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:22.552959 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:22.552908    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pbj6\" (UniqueName: \"kubernetes.io/projected/3a7a0d36-7039-4782-9835-c159ac1e836e-kube-api-access-6pbj6\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:23.265122 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.265087    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj" event={"ID":"3a7a0d36-7039-4782-9835-c159ac1e836e","Type":"ContainerDied","Data":"ef5db7d350e694a8a11b19a1a0f2459c1b810abab82c774fc0a4661065380fc0"}
Oct 29 23:32:23.265122 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.265109    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"
Oct 29 23:32:23.265342 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.265136    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef5db7d350e694a8a11b19a1a0f2459c1b810abab82c774fc0a4661065380fc0"
Oct 29 23:32:23.846084 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.846049    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"]
Oct 29 23:32:23.846838 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.846819    6040 cpu_manager.go:414] "containerMap: removing container" podUID="3a7a0d36-7039-4782-9835-c159ac1e836e" containerName="revoke-keys"
Oct 29 23:32:23.846974 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.846839    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a7a0d36-7039-4782-9835-c159ac1e836e" containerName="revoke-keys"
Oct 29 23:32:23.846974 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.846969    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a7a0d36-7039-4782-9835-c159ac1e836e" containerName="revoke-keys"
Oct 29 23:32:23.850502 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.850480    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:23.865528 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.865507    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-2a53c9-openshift-default-dockercfg-8qsfl\""
Oct 29 23:32:23.878001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.877972    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"]
Oct 29 23:32:23.969176 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.969140    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:23.969337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.969186    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:23.969337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.969260    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:23.969429 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.969378    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:23.969504 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.969481    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbwbn\" (UniqueName: \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-kube-api-access-xbwbn\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:23.969552 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.969534    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:23.969601 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.969580    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:23.969663 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.969650    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:23.969703 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:23.969682    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.070495 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.070457    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.070671 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.070510    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwbn\" (UniqueName: \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-kube-api-access-xbwbn\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.070671 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.070536    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.070863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.070754    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.070863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.070791    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.070863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.070821    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.071023 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.070958    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.071023 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.070970    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.071023 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.070995    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.071183 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.071020    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.071183 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.071073    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.071290 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.071252    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.071329 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.071304    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.071586 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.071563    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.073206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.073177    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.073315 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.073266    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.094136 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.094111    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.094308 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.094290    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbwbn\" (UniqueName: \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-kube-api-access-xbwbn\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.163604 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.163571    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:24.514527 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:32:24.514495    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e71c48_e2a9_4c91_bf79_b5c5baef9c89.slice/crio-1c9f5d07b92d863717fe8dbd169c402a1edb7449f3d13c08fae6c6b2fd176931 WatchSource:0}: Error finding container 1c9f5d07b92d863717fe8dbd169c402a1edb7449f3d13c08fae6c6b2fd176931: Status 404 returned error can't find the container with id 1c9f5d07b92d863717fe8dbd169c402a1edb7449f3d13c08fae6c6b2fd176931
Oct 29 23:32:24.519969 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.519915    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:32:24.521420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.520022    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:32:24.521420 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.520091    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:32:24.525988 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:24.524501    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"]
Oct 29 23:32:25.281079 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:25.281037    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" event={"ID":"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89","Type":"ContainerStarted","Data":"9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29"}
Oct 29 23:32:25.281469 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:25.281102    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" event={"ID":"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89","Type":"ContainerStarted","Data":"1c9f5d07b92d863717fe8dbd169c402a1edb7449f3d13c08fae6c6b2fd176931"}
Oct 29 23:32:25.332356 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:25.332295    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" podStartSLOduration=2.332278198 podStartE2EDuration="2.332278198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:32:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:32:25.329619642 +0000 UTC m=+1125.024728548" watchObservedRunningTime="2026-10-29 23:32:25.332278198 +0000 UTC m=+1125.027387151"
Oct 29 23:32:26.164029 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:26.163989    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:26.165580 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:26.165555    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:32:26.165651 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:26.165605    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" podUID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 29 23:32:27.164858 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:27.164823    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:32:27.165441 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:27.164883    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" podUID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 29 23:32:28.164924 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:28.164888    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:32:28.165359 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:28.164938    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" podUID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 29 23:32:29.164355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:29.164319    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:32:29.164549 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:29.164372    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" podUID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 29 23:32:29.783131 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:32:29.783089    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:32:30.164121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:30.164083    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:32:30.164300 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:30.164137    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" podUID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 29 23:32:31.164391 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:31.164343    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:32:31.164987 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:31.164431    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" podUID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 29 23:32:32.168616 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:32.168583    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:32.169024 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:32.168920    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:32.169700 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:32.169684    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:32:36.985764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:36.975352    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:37.033564 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.032874    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:37.133918 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.133853    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:37.262010 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.261896    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 29 23:32:37.310764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.307857    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": context deadline exceeded"
Oct 29 23:32:37.310764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.308804    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:37.446764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.422266    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:32:37.446764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.435140    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"]
Oct 29 23:32:37.529324 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.529222    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"]
Oct 29 23:32:37.782339 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.782225    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:37.833314 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.833254    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:32:37.863929 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.863871    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:37.920959 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:37.920895    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:38.009051 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.009009    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:38.011619 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.009895    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:32:38.118820 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.090004    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 29 23:32:38.140015 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.139967    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:38.160322 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.151299    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:32:38.209915 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.209879    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:32:38.302886 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.302825    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:38.359808 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.325195    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": context deadline exceeded"
Oct 29 23:32:38.359808 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.325240    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:38.387073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.387042    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:32:38.392947 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.392911    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:32:38.393140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.392971    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:32:38.548768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.546872    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:38.636419 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.636384    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:32:38.761722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.761624    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:32:38.762513 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.762490    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:32:38.878764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.866724    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:38.878764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.868277    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:32:38.904763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.889760    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:38.904763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.890274    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:32:38.991764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.967833    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:38.991764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:38.968402    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:32:39.012931 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.012815    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:39.149763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.148836    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:39.161889 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.161832    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:39.369563 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.369464    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:39.554032 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.553982    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:32:39.768921 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.768873    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:32:39.825933 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.825880    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:39.846510 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.846466    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:39.875774 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.870218    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:39.896762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.893203    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:39.975479 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.975433    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:39.997293 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:39.997243    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:40.020383 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.020284    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:32:40.046345 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.046317    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:32:40.193881 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.193835    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:40.380717 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.370965    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:40.387556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.387501    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:32:40.562918 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.562114    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:32:40.598323 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.598289    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:32:40.682540 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.682416    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:32:40.787710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.787676    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:32:40.793457 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.792579    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:32:40.808818 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.794149    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:32:40.832843 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:40.828033    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:32:46.088123 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.088088    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"]
Oct 29 23:32:46.094038 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.094015    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.113173 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.113149    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"]
Oct 29 23:32:46.120851 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.120827    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-aa7e96\"/\"distinct-w4-aa7e96-kserve-self-signed-certs\""
Oct 29 23:32:46.162110 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.162081    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tls-certs\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.162259 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.162128    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j5ln\" (UniqueName: \"kubernetes.io/projected/dc92da1f-4308-4050-8c1f-7a926c5eae0f-kube-api-access-7j5ln\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.162337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.162317    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tmp-dir\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.162385 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.162367    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-home\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.162437 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.162385    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-dshm\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.162551 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.162529    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-model-cache\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.263653 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.263615    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tmp-dir\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.264053 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.263673    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-home\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.264053 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.263693    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-dshm\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.264053 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.263749    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-model-cache\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.264053 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.263829    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tls-certs\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.264053 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.263880    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j5ln\" (UniqueName: \"kubernetes.io/projected/dc92da1f-4308-4050-8c1f-7a926c5eae0f-kube-api-access-7j5ln\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.264716 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.264678    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tmp-dir\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.265180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.265157    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-home\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.265582 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.265559    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-model-cache\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.276105 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.276072    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-dshm\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.293430 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.293358    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tls-certs\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.309555 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.305396    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j5ln\" (UniqueName: \"kubernetes.io/projected/dc92da1f-4308-4050-8c1f-7a926c5eae0f-kube-api-access-7j5ln\") pod \"distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.409178 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.409138    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:46.762125 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:32:46.762088    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc92da1f_4308_4050_8c1f_7a926c5eae0f.slice/crio-5c94468d30e385b9c0888682b58814ab56d3e5c0f7a4441eb0d4217e6d88210a WatchSource:0}: Error finding container 5c94468d30e385b9c0888682b58814ab56d3e5c0f7a4441eb0d4217e6d88210a: Status 404 returned error can't find the container with id 5c94468d30e385b9c0888682b58814ab56d3e5c0f7a4441eb0d4217e6d88210a
Oct 29 23:32:46.762322 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:46.762303    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"]
Oct 29 23:32:47.336465 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:47.336427    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" event={"ID":"dc92da1f-4308-4050-8c1f-7a926c5eae0f","Type":"ContainerStarted","Data":"1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e"}
Oct 29 23:32:47.336915 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:47.336476    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" event={"ID":"dc92da1f-4308-4050-8c1f-7a926c5eae0f","Type":"ContainerStarted","Data":"5c94468d30e385b9c0888682b58814ab56d3e5c0f7a4441eb0d4217e6d88210a"}
Oct 29 23:32:47.384122 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:47.384069    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podStartSLOduration=1.3840552879999999 podStartE2EDuration="1.384055288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:32:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:32:47.381705551 +0000 UTC m=+1147.076814429" watchObservedRunningTime="2026-10-29 23:32:47.384055288 +0000 UTC m=+1147.079164167"
Oct 29 23:32:47.409443 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:47.409410    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:50.553389 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:50.553352    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"]
Oct 29 23:32:50.574748 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:50.574697    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"]
Oct 29 23:32:52.568925 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:52.568892    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:32:52.718040 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:52.718007    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main" containerID="cri-o://e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7" gracePeriod=45
Oct 29 23:32:53.000087 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.000052    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:32:53.158826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.158259    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-tmp-dir\") pod \"2a3d9093-7037-44e8-a741-743b6b1d4665\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") "
Oct 29 23:32:53.158826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.158377    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a3d9093-7037-44e8-a741-743b6b1d4665-tls-certs\" (UniqueName: \"kubernetes.io/secret/2a3d9093-7037-44e8-a741-743b6b1d4665-tls-certs\") pod \"2a3d9093-7037-44e8-a741-743b6b1d4665\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") "
Oct 29 23:32:53.158826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.158522    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-dshm\") pod \"2a3d9093-7037-44e8-a741-743b6b1d4665\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") "
Oct 29 23:32:53.158826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.158598    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a3d9093-7037-44e8-a741-743b6b1d4665-kube-api-access-q6465\" (UniqueName: \"kubernetes.io/projected/2a3d9093-7037-44e8-a741-743b6b1d4665-kube-api-access-q6465\") pod \"2a3d9093-7037-44e8-a741-743b6b1d4665\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") "
Oct 29 23:32:53.158826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.158660    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-model-cache\") pod \"2a3d9093-7037-44e8-a741-743b6b1d4665\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") "
Oct 29 23:32:53.158826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.158769    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-home\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-home\") pod \"2a3d9093-7037-44e8-a741-743b6b1d4665\" (UID: \"2a3d9093-7037-44e8-a741-743b6b1d4665\") "
Oct 29 23:32:53.160556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.160258    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-tmp-dir" pod "2a3d9093-7037-44e8-a741-743b6b1d4665" (UID: "2a3d9093-7037-44e8-a741-743b6b1d4665"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:32:53.160556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.160286    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-home" pod "2a3d9093-7037-44e8-a741-743b6b1d4665" (UID: "2a3d9093-7037-44e8-a741-743b6b1d4665"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:32:53.160556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.160278    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-model-cache" pod "2a3d9093-7037-44e8-a741-743b6b1d4665" (UID: "2a3d9093-7037-44e8-a741-743b6b1d4665"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:32:53.163070 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.163039    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-dshm" pod "2a3d9093-7037-44e8-a741-743b6b1d4665" (UID: "2a3d9093-7037-44e8-a741-743b6b1d4665"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:32:53.163200 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.163148    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a3d9093-7037-44e8-a741-743b6b1d4665-tls-certs" pod "2a3d9093-7037-44e8-a741-743b6b1d4665" (UID: "2a3d9093-7037-44e8-a741-743b6b1d4665"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:32:53.163200 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.163177    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a3d9093-7037-44e8-a741-743b6b1d4665-kube-api-access-q6465" pod "2a3d9093-7037-44e8-a741-743b6b1d4665" (UID: "2a3d9093-7037-44e8-a741-743b6b1d4665"). InnerVolumeSpecName "kube-api-access-q6465". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:32:53.260833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.260788    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:53.260833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.260823    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:53.260833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.260836    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a3d9093-7037-44e8-a741-743b6b1d4665-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:53.261080 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.260850    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:53.261080 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.260863    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6465\" (UniqueName: \"kubernetes.io/projected/2a3d9093-7037-44e8-a741-743b6b1d4665-kube-api-access-q6465\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:53.261080 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.260875    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a3d9093-7037-44e8-a741-743b6b1d4665-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:53.460363 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.460256    6040 generic.go:356] "Generic (PLEG): container finished" podID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerID="e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7" exitCode=0
Oct 29 23:32:53.460555 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.460516    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"
Oct 29 23:32:53.463601 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.460523    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" event={"ID":"2a3d9093-7037-44e8-a741-743b6b1d4665","Type":"ContainerDied","Data":"e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7"}
Oct 29 23:32:53.463601 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.463013    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh" event={"ID":"2a3d9093-7037-44e8-a741-743b6b1d4665","Type":"ContainerDied","Data":"68fab848718f28828d804e470c983df1f246cab39d05a7ed93929f118d5d0138"}
Oct 29 23:32:53.463601 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.463062    6040 scope.go:122] "RemoveContainer" containerID="e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7"
Oct 29 23:32:53.485963 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.485939    6040 scope.go:122] "RemoveContainer" containerID="e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7"
Oct 29 23:32:53.486266 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:32:53.486241    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7\": container with ID starting with e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7 not found: ID does not exist" containerID="e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7"
Oct 29 23:32:53.486346 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.486277    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7"} err="failed to get container status \"e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7\": rpc error: code = NotFound desc = could not find container \"e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7\": container with ID starting with e36a950bada1034b4ba84d7dc564af2b0aa07bb764a8ccbff5933bdac0e0cea7 not found: ID does not exist"
Oct 29 23:32:53.520238 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.520206    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"]
Oct 29 23:32:53.530232 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:53.530199    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/simulator-w5-53c44e-kserve-697b6d6464-tp7dh"]
Oct 29 23:32:54.319762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:54.319093    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main" containerID="cri-o://f4f9268fde1e7870e6206309f13fc4b6f890ace147ea0b4ef80f01f2cddc5bf6" gracePeriod=44
Oct 29 23:32:54.436788 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:54.436087    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" path="/var/lib/kubelet/pods/2a3d9093-7037-44e8-a741-743b6b1d4665/volumes"
Oct 29 23:32:54.473131 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:54.472944    6040 generic.go:356] "Generic (PLEG): container finished" podID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerID="f4f9268fde1e7870e6206309f13fc4b6f890ace147ea0b4ef80f01f2cddc5bf6" exitCode=0
Oct 29 23:32:54.473131 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:54.473066    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" event={"ID":"92ba9750-151b-4341-a4c4-7751e47dd8b5","Type":"ContainerDied","Data":"f4f9268fde1e7870e6206309f13fc4b6f890ace147ea0b4ef80f01f2cddc5bf6"}
Oct 29 23:32:55.235328 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.220049    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"]
Oct 29 23:32:55.235328 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.230755    6040 cpu_manager.go:414] "containerMap: removing container" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main"
Oct 29 23:32:55.235328 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.230782    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main"
Oct 29 23:32:55.235328 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.231027    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a3d9093-7037-44e8-a741-743b6b1d4665" containerName="main"
Oct 29 23:32:55.253835 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.235953    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:32:55.278847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.275134    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.307815 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.307001    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"]
Oct 29 23:32:55.346761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.346432    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-model-cache\") pod \"92ba9750-151b-4341-a4c4-7751e47dd8b5\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") "
Oct 29 23:32:55.346761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.346524    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-tmp-dir\") pod \"92ba9750-151b-4341-a4c4-7751e47dd8b5\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") "
Oct 29 23:32:55.372167 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.359069    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-tmp-dir" pod "92ba9750-151b-4341-a4c4-7751e47dd8b5" (UID: "92ba9750-151b-4341-a4c4-7751e47dd8b5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:32:55.372167 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.359380    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-model-cache" pod "92ba9750-151b-4341-a4c4-7751e47dd8b5" (UID: "92ba9750-151b-4341-a4c4-7751e47dd8b5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:32:55.372167 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.359481    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92ba9750-151b-4341-a4c4-7751e47dd8b5-tls-certs\" (UniqueName: \"kubernetes.io/secret/92ba9750-151b-4341-a4c4-7751e47dd8b5-tls-certs\") pod \"92ba9750-151b-4341-a4c4-7751e47dd8b5\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") "
Oct 29 23:32:55.372167 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.360113    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-dshm\") pod \"92ba9750-151b-4341-a4c4-7751e47dd8b5\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") "
Oct 29 23:32:55.372167 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.364915    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92ba9750-151b-4341-a4c4-7751e47dd8b5-kube-api-access-5rczp\" (UniqueName: \"kubernetes.io/projected/92ba9750-151b-4341-a4c4-7751e47dd8b5-kube-api-access-5rczp\") pod \"92ba9750-151b-4341-a4c4-7751e47dd8b5\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") "
Oct 29 23:32:55.372167 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.372001    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-home\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-home\") pod \"92ba9750-151b-4341-a4c4-7751e47dd8b5\" (UID: \"92ba9750-151b-4341-a4c4-7751e47dd8b5\") "
Oct 29 23:32:55.372599 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.372344    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.372599 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.372414    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.372599 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.372504    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.372599 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.372566    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.428910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.372603    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.428910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.372665    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.428910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.372690    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpwwg\" (UniqueName: \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-kube-api-access-jpwwg\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.428910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.372728    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.428910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.372789    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.428910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.372993    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:55.428910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.373009    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:55.428910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.420127    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-home" pod "92ba9750-151b-4341-a4c4-7751e47dd8b5" (UID: "92ba9750-151b-4341-a4c4-7751e47dd8b5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:32:55.502948 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.479236    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.502948 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.479431    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.502948 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.479493    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.502948 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.479578    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.502948 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.479633    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.502948 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.479671    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.502948 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.479725    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.528313 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.528237    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpwwg\" (UniqueName: \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-kube-api-access-jpwwg\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.528620 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.528571    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.528982 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.528888    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:55.528982 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.501909    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.528982 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.502175    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.529297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.489955    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ba9750-151b-4341-a4c4-7751e47dd8b5-kube-api-access-5rczp" pod "92ba9750-151b-4341-a4c4-7751e47dd8b5" (UID: "92ba9750-151b-4341-a4c4-7751e47dd8b5"). InnerVolumeSpecName "kube-api-access-5rczp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:32:55.529757 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.490671    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.530310 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.530256    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.530310 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.491360    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.537006 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.536965    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-dshm" pod "92ba9750-151b-4341-a4c4-7751e47dd8b5" (UID: "92ba9750-151b-4341-a4c4-7751e47dd8b5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:32:55.618854 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.578408    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.689301 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.640326    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.689301 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.641499    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rczp\" (UniqueName: \"kubernetes.io/projected/92ba9750-151b-4341-a4c4-7751e47dd8b5-kube-api-access-5rczp\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:55.689301 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.641525    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ba9750-151b-4341-a4c4-7751e47dd8b5-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:55.689301 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.643050    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92ba9750-151b-4341-a4c4-7751e47dd8b5-tls-certs" pod "92ba9750-151b-4341-a4c4-7751e47dd8b5" (UID: "92ba9750-151b-4341-a4c4-7751e47dd8b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:32:55.780642 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.770173    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ba9750-151b-4341-a4c4-7751e47dd8b5-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:32:55.780642 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.770882    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:55.780642 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.771582    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:55.796996 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.795828    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 29 23:32:55.824125 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.822869    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:55.873140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.837374    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" event={"ID":"92ba9750-151b-4341-a4c4-7751e47dd8b5","Type":"ContainerDied","Data":"8da0a21407307cdd58bb229f375c822238da7b50de62495e14f0a92e58edbe81"}
Oct 29 23:32:55.873140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.837439    6040 scope.go:122] "RemoveContainer" containerID="f4f9268fde1e7870e6206309f13fc4b6f890ace147ea0b4ef80f01f2cddc5bf6"
Oct 29 23:32:55.873140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.837601    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"
Oct 29 23:32:55.873140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.869862    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Oct 29 23:32:55.895309 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:55.895261    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpwwg\" (UniqueName: \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-kube-api-access-jpwwg\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:56.015786 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.013570    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"]
Oct 29 23:32:56.025786 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.025334    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-2-w5-53c44e-kserve-86c8f478bc-rg2sr"]
Oct 29 23:32:56.046995 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.045824    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:32:56.071000 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.070843    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:56.076757 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.075830    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:56.138435 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.138101    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:32:56.196937 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.195822    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:56.271164 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.270840    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": context deadline exceeded"
Oct 29 23:32:56.414291 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.412975    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:32:56.458768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.450123    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" path="/var/lib/kubelet/pods/92ba9750-151b-4341-a4c4-7751e47dd8b5/volumes"
Oct 29 23:32:56.568851 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.568791    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": context deadline exceeded"
Oct 29 23:32:56.580031 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.579977    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:32:56.790987 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.790870    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:56.792073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.792047    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:32:56.799923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.799883    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:56.847827 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.801895    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:56.937184 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.849022    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:32:56.952797 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.871129    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:32:56.953572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:56.953542    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:32:57.055040 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.054997    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:57.055626 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.055604    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:32:57.110485 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.081873    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:57.110485 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.082305    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:32:57.110485 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.101301    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"]
Oct 29 23:32:57.110485 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.101399    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:32:57.209372 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.160963    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:32:57.209372 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.161077    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:32:57.209372 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.161125    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:32:57.503765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.496410    6040 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:57.588486 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.588439    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:57.591621 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.591594    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:32:57.902758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.901157    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:32:57.902758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.902510    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 29 23:32:57.945767 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.921258    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 29 23:32:57.946938 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.946897    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:58.141764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:57.959444    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:32:58.141764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:58.060904    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:58.141764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:58.066219    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:32:58.141764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:58.091822    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:58.241819 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:58.211981    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:32:58.328555 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:58.328526    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:32:58.374962 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:58.372350    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" event={"ID":"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b","Type":"ContainerStarted","Data":"51a296b134afe0a0d5a889f797f9870c4d1f30f371ced7e983ec760a5ca4e419"}
Oct 29 23:32:58.464954 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:58.464925    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:32:58.508245 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:58.508148    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:32:58.613660 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:58.613632    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:32:58.623818 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:58.623794    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:32:59.379972 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:59.379929    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" event={"ID":"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b","Type":"ContainerStarted","Data":"cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653"}
Oct 29 23:32:59.430244 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:32:59.430190    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" podStartSLOduration=4.430175816 podStartE2EDuration="4.430175816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:32:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:32:59.428662364 +0000 UTC m=+1159.123771245" watchObservedRunningTime="2026-10-29 23:32:59.430175816 +0000 UTC m=+1159.125284695"
Oct 29 23:32:59.782193 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:32:59.782158    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:33:00.139858 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:00.138954    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:33:00.146787 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:00.144521    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:33:00.146787 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:00.144581    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 29 23:33:01.139628 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:01.139595    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:33:01.140067 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:01.139646    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 29 23:33:02.139124 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:02.139009    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:33:02.139331 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:02.139117    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 29 23:33:03.139613 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.139572    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:33:03.140056 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.139625    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 29 23:33:03.931408 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.931369    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"]
Oct 29 23:33:03.932201 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.932173    6040 cpu_manager.go:414] "containerMap: removing container" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main"
Oct 29 23:33:03.932201 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.932198    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main"
Oct 29 23:33:03.932412 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.932337    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="92ba9750-151b-4341-a4c4-7751e47dd8b5" containerName="main"
Oct 29 23:33:03.941164 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.941134    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:03.952446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.952413    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"]
Oct 29 23:33:03.959100 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.959075    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:33:03.959264 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.959192    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:33:03.989335 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.989300    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2b5h\" (UniqueName: \"kubernetes.io/projected/c22b1162-a7c9-49ed-9426-1a8804fb323f-kube-api-access-l2b5h\") pod \"maas-api-revoke-sub-e41943f9-772kz\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:03.989533 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.989430    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e41943f9-772kz\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:03.989598 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:03.989573    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e41943f9-772kz\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:04.090994 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:04.090954    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2b5h\" (UniqueName: \"kubernetes.io/projected/c22b1162-a7c9-49ed-9426-1a8804fb323f-kube-api-access-l2b5h\") pod \"maas-api-revoke-sub-e41943f9-772kz\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:04.091190 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:04.091080    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e41943f9-772kz\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:04.091190 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:04.091169    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e41943f9-772kz\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:04.091900 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:04.091879    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e41943f9-772kz\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:04.093412 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:04.093388    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e41943f9-772kz\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:04.131242 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:04.131207    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2b5h\" (UniqueName: \"kubernetes.io/projected/c22b1162-a7c9-49ed-9426-1a8804fb323f-kube-api-access-l2b5h\") pod \"maas-api-revoke-sub-e41943f9-772kz\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:04.139869 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:04.139839    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:33:04.140245 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:04.139892    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 29 23:33:04.258620 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:04.258538    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:04.590097 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:04.590066    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"]
Oct 29 23:33:04.591194 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:33:04.591166    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22b1162_a7c9_49ed_9426_1a8804fb323f.slice/crio-c3a0a024573cd13dbe7358e6edf069dfc8e61f0c0ae9a9244cd301f9e701243a WatchSource:0}: Error finding container c3a0a024573cd13dbe7358e6edf069dfc8e61f0c0ae9a9244cd301f9e701243a: Status 404 returned error can't find the container with id c3a0a024573cd13dbe7358e6edf069dfc8e61f0c0ae9a9244cd301f9e701243a
Oct 29 23:33:05.139812 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:05.139780    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:33:05.139978 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:05.139836    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 29 23:33:05.467868 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:05.467602    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz" event={"ID":"c22b1162-a7c9-49ed-9426-1a8804fb323f","Type":"ContainerStarted","Data":"d888c82fbd67905327ac4a5d3bd9325b9328a3856fde6a60c7e2c788321c9f6b"}
Oct 29 23:33:05.467868 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:05.467669    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz" event={"ID":"c22b1162-a7c9-49ed-9426-1a8804fb323f","Type":"ContainerStarted","Data":"c3a0a024573cd13dbe7358e6edf069dfc8e61f0c0ae9a9244cd301f9e701243a"}
Oct 29 23:33:05.515763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:05.514219    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz" podStartSLOduration=2.514202729 podStartE2EDuration="2.514202729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:05.511532208 +0000 UTC m=+1165.206641089" watchObservedRunningTime="2026-10-29 23:33:05.514202729 +0000 UTC m=+1165.209311610"
Oct 29 23:33:05.577720 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:05.577676    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main" containerID="cri-o://dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326" gracePeriod=45
Oct 29 23:33:05.595060 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:05.595017    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main" containerID="cri-o://da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24" gracePeriod=45
Oct 29 23:33:05.751441 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:05.751349    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": dial tcp 10.133.0.48:8000: connect: connection refused"
Oct 29 23:33:05.762883 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:05.762808    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": dial tcp 10.133.0.61:8000: connect: connection refused"
Oct 29 23:33:05.910928 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:05.910890    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:33:05.915723 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:05.915697    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:33:06.056176 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.056092    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d02fd950-ae55-44f5-9fd9-56fa42db262b-tls-certs\" (UniqueName: \"kubernetes.io/secret/d02fd950-ae55-44f5-9fd9-56fa42db262b-tls-certs\") pod \"d02fd950-ae55-44f5-9fd9-56fa42db262b\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") "
Oct 29 23:33:06.056176 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.056151    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fd45077-7c35-4519-8a8f-554c5fbf4128-tls-certs\" (UniqueName: \"kubernetes.io/secret/6fd45077-7c35-4519-8a8f-554c5fbf4128-tls-certs\") pod \"6fd45077-7c35-4519-8a8f-554c5fbf4128\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") "
Oct 29 23:33:06.056406 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.056192    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-dshm\") pod \"6fd45077-7c35-4519-8a8f-554c5fbf4128\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") "
Oct 29 23:33:06.056406 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.056265    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fd45077-7c35-4519-8a8f-554c5fbf4128-kube-api-access-q9pqx\" (UniqueName: \"kubernetes.io/projected/6fd45077-7c35-4519-8a8f-554c5fbf4128-kube-api-access-q9pqx\") pod \"6fd45077-7c35-4519-8a8f-554c5fbf4128\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") "
Oct 29 23:33:06.056688 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.056659    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-dshm\") pod \"d02fd950-ae55-44f5-9fd9-56fa42db262b\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") "
Oct 29 23:33:06.056786 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.056725    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-model-cache\") pod \"6fd45077-7c35-4519-8a8f-554c5fbf4128\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") "
Oct 29 23:33:06.056859 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.056837    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-tmp-dir\") pod \"6fd45077-7c35-4519-8a8f-554c5fbf4128\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") "
Oct 29 23:33:06.056919 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.056900    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-tmp-dir\") pod \"d02fd950-ae55-44f5-9fd9-56fa42db262b\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") "
Oct 29 23:33:06.056975 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.056935    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-home\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-home\") pod \"6fd45077-7c35-4519-8a8f-554c5fbf4128\" (UID: \"6fd45077-7c35-4519-8a8f-554c5fbf4128\") "
Oct 29 23:33:06.057031 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.056977    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d02fd950-ae55-44f5-9fd9-56fa42db262b-kube-api-access-x5tnj\" (UniqueName: \"kubernetes.io/projected/d02fd950-ae55-44f5-9fd9-56fa42db262b-kube-api-access-x5tnj\") pod \"d02fd950-ae55-44f5-9fd9-56fa42db262b\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") "
Oct 29 23:33:06.057031 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.057006    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-model-cache\") pod \"d02fd950-ae55-44f5-9fd9-56fa42db262b\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") "
Oct 29 23:33:06.057145 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.057049    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-home\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-home\") pod \"d02fd950-ae55-44f5-9fd9-56fa42db262b\" (UID: \"d02fd950-ae55-44f5-9fd9-56fa42db262b\") "
Oct 29 23:33:06.057774 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.057717    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-tmp-dir" pod "d02fd950-ae55-44f5-9fd9-56fa42db262b" (UID: "d02fd950-ae55-44f5-9fd9-56fa42db262b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:06.057937 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.057919    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-tmp-dir" pod "6fd45077-7c35-4519-8a8f-554c5fbf4128" (UID: "6fd45077-7c35-4519-8a8f-554c5fbf4128"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:06.057937 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.057932    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-home" pod "6fd45077-7c35-4519-8a8f-554c5fbf4128" (UID: "6fd45077-7c35-4519-8a8f-554c5fbf4128"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:06.058088 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.057930    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-model-cache" pod "d02fd950-ae55-44f5-9fd9-56fa42db262b" (UID: "d02fd950-ae55-44f5-9fd9-56fa42db262b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:06.058214 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.058190    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-home" pod "d02fd950-ae55-44f5-9fd9-56fa42db262b" (UID: "d02fd950-ae55-44f5-9fd9-56fa42db262b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:06.058297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.058261    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-model-cache" pod "6fd45077-7c35-4519-8a8f-554c5fbf4128" (UID: "6fd45077-7c35-4519-8a8f-554c5fbf4128"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:06.058362 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.058341    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.058421 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.058369    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.058421 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.058385    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.058421 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.058400    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.059681 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.059659    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-dshm" pod "6fd45077-7c35-4519-8a8f-554c5fbf4128" (UID: "6fd45077-7c35-4519-8a8f-554c5fbf4128"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:06.059681 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.059669    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd45077-7c35-4519-8a8f-554c5fbf4128-kube-api-access-q9pqx" pod "6fd45077-7c35-4519-8a8f-554c5fbf4128" (UID: "6fd45077-7c35-4519-8a8f-554c5fbf4128"). InnerVolumeSpecName "kube-api-access-q9pqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:06.060998 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.060977    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d02fd950-ae55-44f5-9fd9-56fa42db262b-tls-certs" pod "d02fd950-ae55-44f5-9fd9-56fa42db262b" (UID: "d02fd950-ae55-44f5-9fd9-56fa42db262b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:33:06.061104 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.061032    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd45077-7c35-4519-8a8f-554c5fbf4128-tls-certs" pod "6fd45077-7c35-4519-8a8f-554c5fbf4128" (UID: "6fd45077-7c35-4519-8a8f-554c5fbf4128"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:33:06.061104 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.061068    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-dshm" pod "d02fd950-ae55-44f5-9fd9-56fa42db262b" (UID: "d02fd950-ae55-44f5-9fd9-56fa42db262b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:06.061266 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.061241    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d02fd950-ae55-44f5-9fd9-56fa42db262b-kube-api-access-x5tnj" pod "d02fd950-ae55-44f5-9fd9-56fa42db262b" (UID: "d02fd950-ae55-44f5-9fd9-56fa42db262b"). InnerVolumeSpecName "kube-api-access-x5tnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:06.131378 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.131336    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"]
Oct 29 23:33:06.132206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.132182    6040 cpu_manager.go:414] "containerMap: removing container" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main"
Oct 29 23:33:06.132206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.132206    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main"
Oct 29 23:33:06.132395 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.132273    6040 cpu_manager.go:414] "containerMap: removing container" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main"
Oct 29 23:33:06.132395 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.132282    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main"
Oct 29 23:33:06.132500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.132457    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerName="main"
Oct 29 23:33:06.132500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.132478    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerName="main"
Oct 29 23:33:06.137423 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.137399    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.139416 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.139393    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:33:06.139530 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.139447    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 29 23:33:06.157619 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.157524    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-2a53c9\"/\"distinct-2-w1-2a53c9-kserve-self-signed-certs\""
Oct 29 23:33:06.162174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.159760    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5tnj\" (UniqueName: \"kubernetes.io/projected/d02fd950-ae55-44f5-9fd9-56fa42db262b-kube-api-access-x5tnj\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.162174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.159794    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.162174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.159810    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d02fd950-ae55-44f5-9fd9-56fa42db262b-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.162174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.159823    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fd45077-7c35-4519-8a8f-554c5fbf4128-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.162174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.159836    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.162174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.159848    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9pqx\" (UniqueName: \"kubernetes.io/projected/6fd45077-7c35-4519-8a8f-554c5fbf4128-kube-api-access-q9pqx\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.162174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.159862    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d02fd950-ae55-44f5-9fd9-56fa42db262b-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.162174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.159875    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fd45077-7c35-4519-8a8f-554c5fbf4128-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:06.171227 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.171177    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"]
Oct 29 23:33:06.260992 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.260957    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-model-cache\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.261184 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.261019    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-home\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.261184 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.261107    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-dshm\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.261318 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.261185    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07800096-1dc8-41ca-9a09-756653bbf742-tls-certs\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.261318 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.261202    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx6qv\" (UniqueName: \"kubernetes.io/projected/07800096-1dc8-41ca-9a09-756653bbf742-kube-api-access-zx6qv\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.261318 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.261239    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-tmp-dir\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.364360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.362796    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-model-cache\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.364360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.362889    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-home\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.364360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.362939    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-dshm\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.364360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.363011    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07800096-1dc8-41ca-9a09-756653bbf742-tls-certs\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.364360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.363041    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx6qv\" (UniqueName: \"kubernetes.io/projected/07800096-1dc8-41ca-9a09-756653bbf742-kube-api-access-zx6qv\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.364360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.363095    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-tmp-dir\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.364360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.363281    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-model-cache\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.364360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.363575    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-tmp-dir\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.364360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.363935    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-home\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.395958 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.394380    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07800096-1dc8-41ca-9a09-756653bbf742-tls-certs\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.395958 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.394631    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-dshm\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.411558 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.411476    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx6qv\" (UniqueName: \"kubernetes.io/projected/07800096-1dc8-41ca-9a09-756653bbf742-kube-api-access-zx6qv\") pod \"distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.455517 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.455445    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:06.550086 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.550054    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:33:06.553764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.552046    6040 generic.go:356] "Generic (PLEG): container finished" podID="d02fd950-ae55-44f5-9fd9-56fa42db262b" containerID="dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326" exitCode=0
Oct 29 23:33:06.553764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.552175    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" event={"ID":"d02fd950-ae55-44f5-9fd9-56fa42db262b","Type":"ContainerDied","Data":"dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326"}
Oct 29 23:33:06.553764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.552209    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6" event={"ID":"d02fd950-ae55-44f5-9fd9-56fa42db262b","Type":"ContainerDied","Data":"1aa11a5009390b6d3cf6b43006e6ee21c2071371d90be3d17e625c1e6334c83f"}
Oct 29 23:33:06.553764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.552242    6040 scope.go:122] "RemoveContainer" containerID="dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326"
Oct 29 23:33:06.553764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.552429    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"
Oct 29 23:33:06.563458 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.563371    6040 generic.go:356] "Generic (PLEG): container finished" podID="6fd45077-7c35-4519-8a8f-554c5fbf4128" containerID="da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24" exitCode=0
Oct 29 23:33:06.563583 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.563517    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" event={"ID":"6fd45077-7c35-4519-8a8f-554c5fbf4128","Type":"ContainerDied","Data":"da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24"}
Oct 29 23:33:06.563583 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.563557    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp" event={"ID":"6fd45077-7c35-4519-8a8f-554c5fbf4128","Type":"ContainerDied","Data":"f481fcbf37e59d200dafbcd5f91b088d0e1eca79638a653a044b06cd4b7edd5d"}
Oct 29 23:33:06.563715 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.563671    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"
Oct 29 23:33:06.577175 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.577119    6040 generic.go:356] "Generic (PLEG): container finished" podID="c22b1162-a7c9-49ed-9426-1a8804fb323f" containerID="d888c82fbd67905327ac4a5d3bd9325b9328a3856fde6a60c7e2c788321c9f6b" exitCode=0
Oct 29 23:33:06.577518 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.577180    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz" event={"ID":"c22b1162-a7c9-49ed-9426-1a8804fb323f","Type":"ContainerDied","Data":"d888c82fbd67905327ac4a5d3bd9325b9328a3856fde6a60c7e2c788321c9f6b"}
Oct 29 23:33:06.592025 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.591996    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:33:06.593431 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.593203    6040 scope.go:122] "RemoveContainer" containerID="dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326"
Oct 29 23:33:06.593559 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:33:06.593537    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326\": container with ID starting with dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326 not found: ID does not exist" containerID="dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326"
Oct 29 23:33:06.593626 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.593596    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326"} err="failed to get container status \"dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326\": rpc error: code = NotFound desc = could not find container \"dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326\": container with ID starting with dc31252496c2404a2d57741d0e2d7db699efe98c3078d0dd4fd7663af770f326 not found: ID does not exist"
Oct 29 23:33:06.593626 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.593618    6040 scope.go:122] "RemoveContainer" containerID="da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24"
Oct 29 23:33:06.608595 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.608569    6040 scope.go:122] "RemoveContainer" containerID="da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24"
Oct 29 23:33:06.608985 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:33:06.608962    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24\": container with ID starting with da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24 not found: ID does not exist" containerID="da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24"
Oct 29 23:33:06.609096 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.608992    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24"} err="failed to get container status \"da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24\": rpc error: code = NotFound desc = could not find container \"da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24\": container with ID starting with da77fea454a5034e241ca4313220b15a3447dd7f3b29d7faead164e7b5433f24 not found: ID does not exist"
Oct 29 23:33:06.692501 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.692465    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"]
Oct 29 23:33:06.698586 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.698552    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-w3-92f6b3-kserve-769f875d6-tphx6"]
Oct 29 23:33:06.773086 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.773022    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"]
Oct 29 23:33:06.777518 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.777490    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/simulator-w3-92f6b3-kserve-84dd898bb6-54fqp"]
Oct 29 23:33:06.891231 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:06.891200    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"]
Oct 29 23:33:06.892513 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:33:06.892480    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07800096_1dc8_41ca_9a09_756653bbf742.slice/crio-9a195185dcccd263ed1cff2bcd1fe80a51fb6ee9f591025f37c7ee002073abc5 WatchSource:0}: Error finding container 9a195185dcccd263ed1cff2bcd1fe80a51fb6ee9f591025f37c7ee002073abc5: Status 404 returned error can't find the container with id 9a195185dcccd263ed1cff2bcd1fe80a51fb6ee9f591025f37c7ee002073abc5
Oct 29 23:33:07.144302 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.144229    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:33:07.144772 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.144733    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:33:07.145489 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.145469    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:33:07.586228 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.586139    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" event={"ID":"07800096-1dc8-41ca-9a09-756653bbf742","Type":"ContainerStarted","Data":"15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e"}
Oct 29 23:33:07.586228 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.586188    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" event={"ID":"07800096-1dc8-41ca-9a09-756653bbf742","Type":"ContainerStarted","Data":"9a195185dcccd263ed1cff2bcd1fe80a51fb6ee9f591025f37c7ee002073abc5"}
Oct 29 23:33:07.628995 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.628943    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" podStartSLOduration=1.628928069 podStartE2EDuration="1.628928069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:07.626656808 +0000 UTC m=+1167.321765688" watchObservedRunningTime="2026-10-29 23:33:07.628928069 +0000 UTC m=+1167.324036994"
Oct 29 23:33:07.730528 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.730502    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:07.895339 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.895309    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c22b1162-a7c9-49ed-9426-1a8804fb323f-kube-api-access-l2b5h\" (UniqueName: \"kubernetes.io/projected/c22b1162-a7c9-49ed-9426-1a8804fb323f-kube-api-access-l2b5h\") pod \"c22b1162-a7c9-49ed-9426-1a8804fb323f\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") "
Oct 29 23:33:07.895534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.895370    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-xks-ca\") pod \"c22b1162-a7c9-49ed-9426-1a8804fb323f\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") "
Oct 29 23:33:07.895534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.895399    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-openshift-ca\") pod \"c22b1162-a7c9-49ed-9426-1a8804fb323f\" (UID: \"c22b1162-a7c9-49ed-9426-1a8804fb323f\") "
Oct 29 23:33:07.895838 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.895808    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-openshift-ca" pod "c22b1162-a7c9-49ed-9426-1a8804fb323f" (UID: "c22b1162-a7c9-49ed-9426-1a8804fb323f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:33:07.896001 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.895987    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:07.897566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.897537    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c22b1162-a7c9-49ed-9426-1a8804fb323f-kube-api-access-l2b5h" pod "c22b1162-a7c9-49ed-9426-1a8804fb323f" (UID: "c22b1162-a7c9-49ed-9426-1a8804fb323f"). InnerVolumeSpecName "kube-api-access-l2b5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:07.897566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.897535    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-xks-ca" pod "c22b1162-a7c9-49ed-9426-1a8804fb323f" (UID: "c22b1162-a7c9-49ed-9426-1a8804fb323f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:33:07.996831 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.996793    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2b5h\" (UniqueName: \"kubernetes.io/projected/c22b1162-a7c9-49ed-9426-1a8804fb323f-kube-api-access-l2b5h\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:07.996831 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:07.996825    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c22b1162-a7c9-49ed-9426-1a8804fb323f-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:08.425923 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:08.425887    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fd45077-7c35-4519-8a8f-554c5fbf4128" path="/var/lib/kubelet/pods/6fd45077-7c35-4519-8a8f-554c5fbf4128/volumes"
Oct 29 23:33:08.426419 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:08.426402    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d02fd950-ae55-44f5-9fd9-56fa42db262b" path="/var/lib/kubelet/pods/d02fd950-ae55-44f5-9fd9-56fa42db262b/volumes"
Oct 29 23:33:08.456442 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:08.456409    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:08.595966 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:08.595938    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"
Oct 29 23:33:08.596351 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:08.595972    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz" event={"ID":"c22b1162-a7c9-49ed-9426-1a8804fb323f","Type":"ContainerDied","Data":"c3a0a024573cd13dbe7358e6edf069dfc8e61f0c0ae9a9244cd301f9e701243a"}
Oct 29 23:33:08.596351 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:08.596020    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3a0a024573cd13dbe7358e6edf069dfc8e61f0c0ae9a9244cd301f9e701243a"
Oct 29 23:33:09.640231 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.640195    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"]
Oct 29 23:33:09.640821 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.640798    6040 cpu_manager.go:414] "containerMap: removing container" podUID="c22b1162-a7c9-49ed-9426-1a8804fb323f" containerName="revoke-keys"
Oct 29 23:33:09.640821 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.640819    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c22b1162-a7c9-49ed-9426-1a8804fb323f" containerName="revoke-keys"
Oct 29 23:33:09.640981 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.640924    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="c22b1162-a7c9-49ed-9426-1a8804fb323f" containerName="revoke-keys"
Oct 29 23:33:09.645524 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.645507    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.651550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.651527    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"]
Oct 29 23:33:09.655325 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.655309    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.663393 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.663257    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"]
Oct 29 23:33:09.667364 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.667346    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.676908 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.676884    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"]
Oct 29 23:33:09.681979 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.681951    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"]
Oct 29 23:33:09.696176 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.696151    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"]
Oct 29 23:33:09.819950 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.819915    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663e220-9cf3-4217-a320-9330dfaf1012-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.820121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.819973    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.820121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820029    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.820121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820053    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.820121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820074    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rhr5\" (UniqueName: \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-kube-api-access-7rhr5\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.820270 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820123    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.820270 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820184    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.820270 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820241    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.820270 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820262    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.820402 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820344    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.820402 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820382    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.820471 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820404    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663e220-9cf3-4217-a320-9330dfaf1012-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.820471 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820428    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.820539 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820462    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/256870d0-9dce-4e09-8374-9835d54c4f78-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.820539 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820531    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.820619 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820556    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.820619 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820576    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.820619 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820597    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/256870d0-9dce-4e09-8374-9835d54c4f78-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.820715 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820655    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.820770 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820712    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.820770 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820746    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.820868 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820767    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.820868 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820799    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnchz\" (UniqueName: \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-kube-api-access-cnchz\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.820977 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.820955    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.821037 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.821023    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbdq9\" (UniqueName: \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-kube-api-access-gbdq9\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.821072 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.821058    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.821105 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.821073    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.923003 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.922928    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.923003 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.922964    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.923003 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.922987    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.923251 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923017    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.923251 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923066    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.923251 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923120    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.923251 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923155    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663e220-9cf3-4217-a320-9330dfaf1012-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.923251 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923181    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.923571 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923397    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/256870d0-9dce-4e09-8374-9835d54c4f78-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.923571 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923404    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.923571 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923445    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.923571 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923515    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.923571 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923528    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.923863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923588    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.923863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923623    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/256870d0-9dce-4e09-8374-9835d54c4f78-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.923863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923654    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.923863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923686    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.923863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923719    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.923863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923767    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.923863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923793    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnchz\" (UniqueName: \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-kube-api-access-cnchz\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.923863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923832    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.924278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923919    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbdq9\" (UniqueName: \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-kube-api-access-gbdq9\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.924278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923965    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663e220-9cf3-4217-a320-9330dfaf1012-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.924278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.923998    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.924278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.924023    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.924278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.924181    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663e220-9cf3-4217-a320-9330dfaf1012-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.924278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.924202    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/256870d0-9dce-4e09-8374-9835d54c4f78-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.924278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.924228    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.924618 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.924296    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.924618 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.924326    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.924618 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.924371    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.924618 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.924409    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rhr5\" (UniqueName: \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-kube-api-access-7rhr5\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.924860 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.924705    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.924987 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.924968    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.925062 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.925025    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.925316 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.925282    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.925412 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.925367    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.925769 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.925625    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.926278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.925843    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.926278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.925941    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.926418 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.926281    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.926482 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.926439    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.926573 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.926555    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.926912 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.926893    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.927251 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.927233    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663e220-9cf3-4217-a320-9330dfaf1012-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.927581 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.927560    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.927709 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.927687    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.928034 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.928019    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/256870d0-9dce-4e09-8374-9835d54c4f78-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.946825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.946798    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.947682 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.947660    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbdq9\" (UniqueName: \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-kube-api-access-gbdq9\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.947817 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.947706    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rhr5\" (UniqueName: \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-kube-api-access-7rhr5\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.947817 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.947714    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnchz\" (UniqueName: \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-kube-api-access-cnchz\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.947940 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.947855    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.948723 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.948705    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:09.959713 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.959691    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:09.971084 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.971063    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:09.986498 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:09.986475    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:10.285046 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.285016    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"]
Oct 29 23:33:10.285178 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:33:10.285148    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9663e220_9cf3_4217_a320_9330dfaf1012.slice/crio-cabb760b63c02688795dccf8a09766025cfe5f0f172a02c69ab40b3666f43c9d WatchSource:0}: Error finding container cabb760b63c02688795dccf8a09766025cfe5f0f172a02c69ab40b3666f43c9d: Status 404 returned error can't find the container with id cabb760b63c02688795dccf8a09766025cfe5f0f172a02c69ab40b3666f43c9d
Oct 29 23:33:10.287685 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.287653    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:10.287790 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.287718    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:10.287838 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.287815    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:10.331255 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.331225    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"]
Oct 29 23:33:10.336894 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.336094    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:10.336894 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.336173    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:10.336894 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.336215    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:10.343226 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.343019    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"]
Oct 29 23:33:10.347262 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:33:10.347237    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d9dcc91_f5e8_47fb_801e_251be89b55d5.slice/crio-9125d86d73ae9ef9da29304915bb1e1b918676cb593a67b449f1285c64f4ba6b WatchSource:0}: Error finding container 9125d86d73ae9ef9da29304915bb1e1b918676cb593a67b449f1285c64f4ba6b: Status 404 returned error can't find the container with id 9125d86d73ae9ef9da29304915bb1e1b918676cb593a67b449f1285c64f4ba6b
Oct 29 23:33:10.350002 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.349963    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:10.350103 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.350048    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:10.350103 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.350092    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:10.615503 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.615406    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" event={"ID":"9663e220-9cf3-4217-a320-9330dfaf1012","Type":"ContainerStarted","Data":"b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834"}
Oct 29 23:33:10.615766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.615713    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" event={"ID":"9663e220-9cf3-4217-a320-9330dfaf1012","Type":"ContainerStarted","Data":"cabb760b63c02688795dccf8a09766025cfe5f0f172a02c69ab40b3666f43c9d"}
Oct 29 23:33:10.619216 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.619191    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" event={"ID":"256870d0-9dce-4e09-8374-9835d54c4f78","Type":"ContainerStarted","Data":"731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4"}
Oct 29 23:33:10.619412 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.619397    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" event={"ID":"256870d0-9dce-4e09-8374-9835d54c4f78","Type":"ContainerStarted","Data":"aed31b23740ff0027d43c0c8f21375b7d0a5d8dfd2714e30e1616a69b6bf5fb5"}
Oct 29 23:33:10.622047 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.622015    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" event={"ID":"4d9dcc91-f5e8-47fb-801e-251be89b55d5","Type":"ContainerStarted","Data":"fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23"}
Oct 29 23:33:10.622209 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.622195    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" event={"ID":"4d9dcc91-f5e8-47fb-801e-251be89b55d5","Type":"ContainerStarted","Data":"9125d86d73ae9ef9da29304915bb1e1b918676cb593a67b449f1285c64f4ba6b"}
Oct 29 23:33:10.666924 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.666854    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" podStartSLOduration=1.666835501 podStartE2EDuration="1.666835501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:10.661319107 +0000 UTC m=+1170.356427988" watchObservedRunningTime="2026-10-29 23:33:10.666835501 +0000 UTC m=+1170.361944383"
Oct 29 23:33:10.711235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.711172    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" podStartSLOduration=1.711152363 podStartE2EDuration="1.711152363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:10.70643596 +0000 UTC m=+1170.401544841" watchObservedRunningTime="2026-10-29 23:33:10.711152363 +0000 UTC m=+1170.406261241"
Oct 29 23:33:10.754251 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.754193    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" podStartSLOduration=1.754177447 podStartE2EDuration="1.754177447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:10.751079857 +0000 UTC m=+1170.446188759" watchObservedRunningTime="2026-10-29 23:33:10.754177447 +0000 UTC m=+1170.449286326"
Oct 29 23:33:10.960994 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.960830    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:10.976782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.975453    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:10.992247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:10.992211    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:11.964762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:11.962895    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 29 23:33:11.964762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:11.962958    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 29 23:33:11.977975 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:11.977941    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 29 23:33:11.978234 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:11.978204    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 29 23:33:11.991431 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:11.991400    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 29 23:33:11.991649 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:11.991628    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 29 23:33:12.400329 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.398277    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"]
Oct 29 23:33:12.420173 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.419027    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.500803 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.500766    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-model-cache\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.501063 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.501044    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-home\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.501327 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.501306    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vskzw\" (UniqueName: \"kubernetes.io/projected/680505f8-f92f-4225-b61f-5b323c23b236-kube-api-access-vskzw\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.504758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.501474    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/680505f8-f92f-4225-b61f-5b323c23b236-tls-certs\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.514767 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.505069    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-dshm\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.515029 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.515002    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-tmp-dir\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.516146 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.516089    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-ae1364\"/\"distinct-2-w2-ae1364-kserve-self-signed-certs\""
Oct 29 23:33:12.595764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.564647    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"]
Oct 29 23:33:12.625410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.625367    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-tmp-dir\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.635331 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.634286    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-tmp-dir\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.635721 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.635698    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-model-cache\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.636312 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.636293    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-home\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.641929 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.641891    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vskzw\" (UniqueName: \"kubernetes.io/projected/680505f8-f92f-4225-b61f-5b323c23b236-kube-api-access-vskzw\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.642657 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.642637    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/680505f8-f92f-4225-b61f-5b323c23b236-tls-certs\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.643023 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.636226    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-model-cache\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.643205 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.636782    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-home\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.643875 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.643431    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-dshm\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.659249 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.659156    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-dshm\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.725697 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.725629    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vskzw\" (UniqueName: \"kubernetes.io/projected/680505f8-f92f-4225-b61f-5b323c23b236-kube-api-access-vskzw\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.766462 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.766382    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/680505f8-f92f-4225-b61f-5b323c23b236-tls-certs\") pod \"distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") " pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.930333 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.930002    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:12.965372 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.965331    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 29 23:33:12.965955 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.965907    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 29 23:33:13.010793 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.992844    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 29 23:33:13.010793 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.992954    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 29 23:33:13.010793 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.993098    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 29 23:33:13.010793 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:12.993127    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 29 23:33:13.631779 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:13.604288    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:13.631779 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:13.606591    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:13.945931 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:13.934035    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:14.024774 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:13.998883    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 29 23:33:14.024774 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:13.998947    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 29 23:33:14.024774 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:13.999382    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 29 23:33:14.024774 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:13.999418    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 29 23:33:14.024774 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.000809    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 29 23:33:14.024774 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.000842    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 29 23:33:14.095035 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.094997    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"]
Oct 29 23:33:14.124762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.099081    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:14.243230 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.215497    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:14.342850 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.342800    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:14.628693 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.612078    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:14.628693 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.625852    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 29 23:33:14.963327 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.961671    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:14.972769 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.972318    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 29 23:33:14.972769 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.972374    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 29 23:33:14.976766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.975194    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 29 23:33:14.976766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:14.975636    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 29 23:33:15.002766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.002055    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 29 23:33:15.002766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.002132    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 29 23:33:15.126759 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.126289    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:15.171408 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.170069    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:33:15.171408 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.170168    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:33:15.171408 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.170232    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:33:15.230177 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.230079    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:33:15.254396 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.254363    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:33:15.312901 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.312857    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:15.570828 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.560457    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:15.641509 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.641440    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 29 23:33:15.642128 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.642089    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 29 23:33:15.656762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.645098    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:15.712764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.708856    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:33:15.802780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:15.802726    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" event={"ID":"680505f8-f92f-4225-b61f-5b323c23b236","Type":"ContainerStarted","Data":"efea6f9776a9caa5075b449f6056d8edbc5dc1831ae957d1a37f25c315b53cd4"}
Oct 29 23:33:16.067760 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.040335    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 29 23:33:16.068123 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.068076    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 29 23:33:16.097761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.079463    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:16.097954 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.097891    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 29 23:33:16.098018 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.097945    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 29 23:33:16.105964 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.098209    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 29 23:33:16.105964 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.105730    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 29 23:33:16.105964 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.105798    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 29 23:33:16.115171 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.112671    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:16.155798 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.135979    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:33:16.155798 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.136411    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:33:16.265873 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.265825    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:33:16.290762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.280143    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:16.610540 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.601332    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:16.638776 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.628897    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:16.658235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.646855    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:16.658235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.647685    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:33:16.658235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.647985    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 29 23:33:16.743766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.742455    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:33:16.743766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.742517    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:16.978762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.970098    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 29 23:33:16.978762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.970157    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 29 23:33:16.978762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.978156    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 29 23:33:16.978762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.978219    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 29 23:33:16.999500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.992941    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 29 23:33:16.999844 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:16.999782    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 29 23:33:17.119995 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.114811    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:17.119995 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.115806    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 29 23:33:17.119995 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.116138    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:33:17.179925 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.179882    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:17.332995 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.332880    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:17.395438 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.395407    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:33:17.455409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.455300    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"]
Oct 29 23:33:17.494608 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.494575    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"]
Oct 29 23:33:17.494920 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.494901    6040 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:33:17.608824 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.602025    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:17.624566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.624536    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:33:17.626009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.625976    6040 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" podUID="07800096-1dc8-41ca-9a09-756653bbf742" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:17.648958 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.648918    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 29 23:33:17.649562 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.649182    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:17.667632 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.649216    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:17.673810 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.673730    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:33:17.694297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.694268    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:33:17.785012 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.784977    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:33:17.860083 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.859999    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:33:17.871870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:17.871841    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:33:18.088927 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.088884    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 29 23:33:18.114849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.107620    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 29 23:33:18.114849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.107685    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 29 23:33:18.114849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.107731    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 29 23:33:18.114849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.107769    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 29 23:33:18.124785 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.115260    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 29 23:33:18.190225 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.189483    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:18.210686 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.210655    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:33:18.975388 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.975348    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 29 23:33:18.975658 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.975623    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 29 23:33:18.986329 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.986287    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" event={"ID":"680505f8-f92f-4225-b61f-5b323c23b236","Type":"ContainerStarted","Data":"b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646"}
Oct 29 23:33:18.987166 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.987141    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 29 23:33:18.987266 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:18.987193    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 29 23:33:19.041200 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:19.041160    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:19.042171 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:19.042149    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:19.043230 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:19.043210    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:33:19.091653 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:19.091586    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" podStartSLOduration=7.09157059 podStartE2EDuration="7.09157059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:19.03837151 +0000 UTC m=+1178.733480389" watchObservedRunningTime="2026-10-29 23:33:19.09157059 +0000 UTC m=+1178.786679468"
Oct 29 23:33:19.931274 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:19.931244    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:19.975811 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:19.975779    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:19.976170 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:19.976153    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:19.976895 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:19.976876    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:33:19.991505 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:19.991486    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:19.991803 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:19.991787    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:19.992534 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:19.992515    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:33:22.930961 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:22.930860    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:23.739395 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:23.714311    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:23.857838 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:23.857797    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"]
Oct 29 23:33:23.892765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:23.858281    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" podUID="6291cdad-e6f2-461f-be7e-d970959ae576" containerName="istio-proxy" containerID="cri-o://a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264" gracePeriod=30
Oct 29 23:33:23.918763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:23.883407    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:33:23.936400 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:23.936361    6040 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" podUID="680505f8-f92f-4225-b61f-5b323c23b236" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:23.995886 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:23.995794    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"]
Oct 29 23:33:23.996332 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:23.996299    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerName="istio-proxy" containerID="cri-o://40455e2620c31d6b8511e2975c950967076425c74b309fdde4e3a52b5ee49196" gracePeriod=30
Oct 29 23:33:24.075824 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:24.075793    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"]
Oct 29 23:33:24.076319 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:24.076284    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" podUID="751a2a59-c2db-498e-8d07-eba400912132" containerName="istio-proxy" containerID="cri-o://94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa" gracePeriod=30
Oct 29 23:33:24.117761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:24.117493    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:24.221766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:24.214018    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:24.306028 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:24.305924    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:24.715684 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:24.715627    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 29 23:33:24.716877 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:24.716852    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:33:24.936172 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:24.936101    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:24.937242 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:24.937217    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:33:24.938530 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:24.936427    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 29 23:33:25.289658 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:25.286354    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:33:25.289658 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:25.288180    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:33:25.304761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:25.290502    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:25.308765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:25.306622    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:33:25.308765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:25.307136    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:33:25.401915 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:25.401864    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:25.670779 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:25.668354    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:25.721796 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:25.717519    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:25.796013 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:25.789810    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:33:25.892769 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:25.877830    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:33:25.973823 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:25.973204    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:33:26.454855 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:26.454811    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 29 23:33:26.551376 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:26.551273    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:26.610364 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:26.610337    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:33:26.610561 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:26.610389    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:33:26.610561 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:26.610418    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:33:26.610561 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:26.610468    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:26.610561 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:26.610497    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:33:26.625652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:26.625624    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:29.788301 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:33:29.788264    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:33:30.641449 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:30.641421    6040 generic.go:356] "Generic (PLEG): container finished" podID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerID="40455e2620c31d6b8511e2975c950967076425c74b309fdde4e3a52b5ee49196" exitCode=0
Oct 29 23:33:30.641599 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:30.641499    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" event={"ID":"696b829d-a17c-48d8-9cbc-ec567e7add16","Type":"ContainerDied","Data":"40455e2620c31d6b8511e2975c950967076425c74b309fdde4e3a52b5ee49196"}
Oct 29 23:33:31.078178 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.078143    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 29 23:33:31.078580 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.078206    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 29 23:33:31.244310 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.244284    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:33:31.348040 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.347959    6040 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 29 23:33:31.348040 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.348019    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" podUID="6291cdad-e6f2-461f-be7e-d970959ae576" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:33:31.379628 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.379589    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-socket\") pod \"696b829d-a17c-48d8-9cbc-ec567e7add16\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") "
Oct 29 23:33:31.379848 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.379673    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-token\" (UniqueName: \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-token\") pod \"696b829d-a17c-48d8-9cbc-ec567e7add16\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") "
Oct 29 23:33:31.379848 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.379810    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/696b829d-a17c-48d8-9cbc-ec567e7add16-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/696b829d-a17c-48d8-9cbc-ec567e7add16-istiod-ca-cert\") pod \"696b829d-a17c-48d8-9cbc-ec567e7add16\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") "
Oct 29 23:33:31.379982 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.379900    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-socket" pod "696b829d-a17c-48d8-9cbc-ec567e7add16" (UID: "696b829d-a17c-48d8-9cbc-ec567e7add16"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:31.380210 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.380184    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-credential-socket" pod "696b829d-a17c-48d8-9cbc-ec567e7add16" (UID: "696b829d-a17c-48d8-9cbc-ec567e7add16"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:31.380297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.380239    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/696b829d-a17c-48d8-9cbc-ec567e7add16-istiod-ca-cert" pod "696b829d-a17c-48d8-9cbc-ec567e7add16" (UID: "696b829d-a17c-48d8-9cbc-ec567e7add16"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:33:31.380297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.380289    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-credential-socket\") pod \"696b829d-a17c-48d8-9cbc-ec567e7add16\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") "
Oct 29 23:33:31.380416 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.380364    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-kube-api-access-g7tjw\" (UniqueName: \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-kube-api-access-g7tjw\") pod \"696b829d-a17c-48d8-9cbc-ec567e7add16\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") "
Oct 29 23:33:31.380467 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.380433    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-data\") pod \"696b829d-a17c-48d8-9cbc-ec567e7add16\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") "
Oct 29 23:33:31.380519 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.380465    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-certs\") pod \"696b829d-a17c-48d8-9cbc-ec567e7add16\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") "
Oct 29 23:33:31.380572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.380544    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-envoy\") pod \"696b829d-a17c-48d8-9cbc-ec567e7add16\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") "
Oct 29 23:33:31.380718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.380692    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-data" pod "696b829d-a17c-48d8-9cbc-ec567e7add16" (UID: "696b829d-a17c-48d8-9cbc-ec567e7add16"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:31.380802 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.380761    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-podinfo\") pod \"696b829d-a17c-48d8-9cbc-ec567e7add16\" (UID: \"696b829d-a17c-48d8-9cbc-ec567e7add16\") "
Oct 29 23:33:31.380952 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.380930    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-certs" pod "696b829d-a17c-48d8-9cbc-ec567e7add16" (UID: "696b829d-a17c-48d8-9cbc-ec567e7add16"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:31.381481 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.381459    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:31.381607 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.381594    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:31.381710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.381696    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:31.381823 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.381808    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/696b829d-a17c-48d8-9cbc-ec567e7add16-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:31.381914 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.381900    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:31.382803 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.382728    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-token" pod "696b829d-a17c-48d8-9cbc-ec567e7add16" (UID: "696b829d-a17c-48d8-9cbc-ec567e7add16"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:31.383989 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.383962    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-envoy" pod "696b829d-a17c-48d8-9cbc-ec567e7add16" (UID: "696b829d-a17c-48d8-9cbc-ec567e7add16"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:31.384103 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.384047    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-podinfo" pod "696b829d-a17c-48d8-9cbc-ec567e7add16" (UID: "696b829d-a17c-48d8-9cbc-ec567e7add16"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:33:31.384428 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.384397    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-kube-api-access-g7tjw" pod "696b829d-a17c-48d8-9cbc-ec567e7add16" (UID: "696b829d-a17c-48d8-9cbc-ec567e7add16"). InnerVolumeSpecName "kube-api-access-g7tjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:31.484211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.484177    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7tjw\" (UniqueName: \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-kube-api-access-g7tjw\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:31.484211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.484210    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:31.484211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.484223    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:31.484505 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.484236    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/696b829d-a17c-48d8-9cbc-ec567e7add16-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:31.651672 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.651637    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f" event={"ID":"696b829d-a17c-48d8-9cbc-ec567e7add16","Type":"ContainerDied","Data":"14909d360525f159ec5d973164552caa31a852e1cde5c9c5ac2e3faf64e0853a"}
Oct 29 23:33:31.651882 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.651697    6040 scope.go:122] "RemoveContainer" containerID="40455e2620c31d6b8511e2975c950967076425c74b309fdde4e3a52b5ee49196"
Oct 29 23:33:31.651882 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.651652    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"
Oct 29 23:33:31.708073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.708038    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"]
Oct 29 23:33:31.719557 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.719528    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-clm7f"]
Oct 29 23:33:31.971685 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:31.971659    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:33:32.105777 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.105726    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-certs\") pod \"751a2a59-c2db-498e-8d07-eba400912132\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") "
Oct 29 23:33:32.106220 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.105842    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/751a2a59-c2db-498e-8d07-eba400912132-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/751a2a59-c2db-498e-8d07-eba400912132-istiod-ca-cert\") pod \"751a2a59-c2db-498e-8d07-eba400912132\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") "
Oct 29 23:33:32.106220 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.105894    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-kube-api-access-chqt6\" (UniqueName: \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-kube-api-access-chqt6\") pod \"751a2a59-c2db-498e-8d07-eba400912132\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") "
Oct 29 23:33:32.106220 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.105970    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-certs" pod "751a2a59-c2db-498e-8d07-eba400912132" (UID: "751a2a59-c2db-498e-8d07-eba400912132"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:32.106220 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.105974    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-envoy\") pod \"751a2a59-c2db-498e-8d07-eba400912132\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") "
Oct 29 23:33:32.106220 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.106053    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/751a2a59-c2db-498e-8d07-eba400912132-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/751a2a59-c2db-498e-8d07-eba400912132-istio-podinfo\") pod \"751a2a59-c2db-498e-8d07-eba400912132\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") "
Oct 29 23:33:32.106220 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.106102    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-socket\") pod \"751a2a59-c2db-498e-8d07-eba400912132\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") "
Oct 29 23:33:32.106559 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.106229    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-credential-socket\") pod \"751a2a59-c2db-498e-8d07-eba400912132\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") "
Oct 29 23:33:32.106559 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.106293    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-data\") pod \"751a2a59-c2db-498e-8d07-eba400912132\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") "
Oct 29 23:33:32.106559 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.106416    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-socket" pod "751a2a59-c2db-498e-8d07-eba400912132" (UID: "751a2a59-c2db-498e-8d07-eba400912132"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:32.106559 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.106452    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-istio-token\" (UniqueName: \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-istio-token\") pod \"751a2a59-c2db-498e-8d07-eba400912132\" (UID: \"751a2a59-c2db-498e-8d07-eba400912132\") "
Oct 29 23:33:32.106790 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.106560    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/751a2a59-c2db-498e-8d07-eba400912132-istiod-ca-cert" pod "751a2a59-c2db-498e-8d07-eba400912132" (UID: "751a2a59-c2db-498e-8d07-eba400912132"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:33:32.106790 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.106617    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-credential-socket" pod "751a2a59-c2db-498e-8d07-eba400912132" (UID: "751a2a59-c2db-498e-8d07-eba400912132"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:32.106790 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.106642    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-data" pod "751a2a59-c2db-498e-8d07-eba400912132" (UID: "751a2a59-c2db-498e-8d07-eba400912132"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:32.107062 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.107042    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/751a2a59-c2db-498e-8d07-eba400912132-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.107152 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.107064    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.107152 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.107078    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.107152 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.107092    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.107152 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.107108    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.108891 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.108804    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-istio-token" pod "751a2a59-c2db-498e-8d07-eba400912132" (UID: "751a2a59-c2db-498e-8d07-eba400912132"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:32.108891 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.108858    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/751a2a59-c2db-498e-8d07-eba400912132-istio-podinfo" pod "751a2a59-c2db-498e-8d07-eba400912132" (UID: "751a2a59-c2db-498e-8d07-eba400912132"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:33:32.109043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.108905    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-kube-api-access-chqt6" pod "751a2a59-c2db-498e-8d07-eba400912132" (UID: "751a2a59-c2db-498e-8d07-eba400912132"). InnerVolumeSpecName "kube-api-access-chqt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:32.109573 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.109552    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-envoy" pod "751a2a59-c2db-498e-8d07-eba400912132" (UID: "751a2a59-c2db-498e-8d07-eba400912132"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:32.208887 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.208807    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/751a2a59-c2db-498e-8d07-eba400912132-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.208887 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.208837    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/751a2a59-c2db-498e-8d07-eba400912132-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.208887 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.208847    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.208887 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.208856    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chqt6\" (UniqueName: \"kubernetes.io/projected/751a2a59-c2db-498e-8d07-eba400912132-kube-api-access-chqt6\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.245139 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.245110    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:33:32.411147 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411113    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6291cdad-e6f2-461f-be7e-d970959ae576-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6291cdad-e6f2-461f-be7e-d970959ae576-istio-podinfo\") pod \"6291cdad-e6f2-461f-be7e-d970959ae576\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") "
Oct 29 23:33:32.411347 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411176    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-data\") pod \"6291cdad-e6f2-461f-be7e-d970959ae576\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") "
Oct 29 23:33:32.411347 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411198    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-istio-token\" (UniqueName: \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-istio-token\") pod \"6291cdad-e6f2-461f-be7e-d970959ae576\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") "
Oct 29 23:33:32.411347 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411338    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-certs\") pod \"6291cdad-e6f2-461f-be7e-d970959ae576\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") "
Oct 29 23:33:32.411516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411387    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-kube-api-access-lt2x9\" (UniqueName: \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-kube-api-access-lt2x9\") pod \"6291cdad-e6f2-461f-be7e-d970959ae576\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") "
Oct 29 23:33:32.411579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411517    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6291cdad-e6f2-461f-be7e-d970959ae576-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6291cdad-e6f2-461f-be7e-d970959ae576-istiod-ca-cert\") pod \"6291cdad-e6f2-461f-be7e-d970959ae576\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") "
Oct 29 23:33:32.411579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411518    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-data" pod "6291cdad-e6f2-461f-be7e-d970959ae576" (UID: "6291cdad-e6f2-461f-be7e-d970959ae576"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:32.411579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411544    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-certs" pod "6291cdad-e6f2-461f-be7e-d970959ae576" (UID: "6291cdad-e6f2-461f-be7e-d970959ae576"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:32.411579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411549    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-socket\") pod \"6291cdad-e6f2-461f-be7e-d970959ae576\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") "
Oct 29 23:33:32.411804 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411604    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-credential-socket\") pod \"6291cdad-e6f2-461f-be7e-d970959ae576\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") "
Oct 29 23:33:32.411804 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411666    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-envoy\") pod \"6291cdad-e6f2-461f-be7e-d970959ae576\" (UID: \"6291cdad-e6f2-461f-be7e-d970959ae576\") "
Oct 29 23:33:32.411804 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411757    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-socket" pod "6291cdad-e6f2-461f-be7e-d970959ae576" (UID: "6291cdad-e6f2-461f-be7e-d970959ae576"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:32.411960 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411868    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6291cdad-e6f2-461f-be7e-d970959ae576-istiod-ca-cert" pod "6291cdad-e6f2-461f-be7e-d970959ae576" (UID: "6291cdad-e6f2-461f-be7e-d970959ae576"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:33:32.411960 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.411918    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-credential-socket" pod "6291cdad-e6f2-461f-be7e-d970959ae576" (UID: "6291cdad-e6f2-461f-be7e-d970959ae576"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:32.412433 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.412399    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6291cdad-e6f2-461f-be7e-d970959ae576-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.412433 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.412429    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.412604 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.412445    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.412604 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.412458    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.412604 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.412473    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.413566 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.413540    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-istio-token" pod "6291cdad-e6f2-461f-be7e-d970959ae576" (UID: "6291cdad-e6f2-461f-be7e-d970959ae576"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:32.413779 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.413758    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-envoy" pod "6291cdad-e6f2-461f-be7e-d970959ae576" (UID: "6291cdad-e6f2-461f-be7e-d970959ae576"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:32.413854 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.413840    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6291cdad-e6f2-461f-be7e-d970959ae576-istio-podinfo" pod "6291cdad-e6f2-461f-be7e-d970959ae576" (UID: "6291cdad-e6f2-461f-be7e-d970959ae576"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:33:32.414038 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.414018    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-kube-api-access-lt2x9" pod "6291cdad-e6f2-461f-be7e-d970959ae576" (UID: "6291cdad-e6f2-461f-be7e-d970959ae576"). InnerVolumeSpecName "kube-api-access-lt2x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:32.426344 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.426315    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" path="/var/lib/kubelet/pods/696b829d-a17c-48d8-9cbc-ec567e7add16/volumes"
Oct 29 23:33:32.513678 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.513568    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.513678 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.513603    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt2x9\" (UniqueName: \"kubernetes.io/projected/6291cdad-e6f2-461f-be7e-d970959ae576-kube-api-access-lt2x9\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.513678 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.513613    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6291cdad-e6f2-461f-be7e-d970959ae576-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.513678 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.513622    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6291cdad-e6f2-461f-be7e-d970959ae576-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:32.661518 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.661482    6040 generic.go:356] "Generic (PLEG): container finished" podID="751a2a59-c2db-498e-8d07-eba400912132" containerID="94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa" exitCode=0
Oct 29 23:33:32.661695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.661530    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"
Oct 29 23:33:32.661695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.661558    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" event={"ID":"751a2a59-c2db-498e-8d07-eba400912132","Type":"ContainerDied","Data":"94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa"}
Oct 29 23:33:32.661695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.661610    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v" event={"ID":"751a2a59-c2db-498e-8d07-eba400912132","Type":"ContainerDied","Data":"4b96b7c2e31ba65837dafb9e6c2bfaa2fb71fb05bb079ae86ed35c9a9c41ac21"}
Oct 29 23:33:32.661695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.661624    6040 scope.go:122] "RemoveContainer" containerID="94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa"
Oct 29 23:33:32.663198 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.663175    6040 generic.go:356] "Generic (PLEG): container finished" podID="6291cdad-e6f2-461f-be7e-d970959ae576" containerID="a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264" exitCode=0
Oct 29 23:33:32.663309 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.663235    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"
Oct 29 23:33:32.663309 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.663294    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" event={"ID":"6291cdad-e6f2-461f-be7e-d970959ae576","Type":"ContainerDied","Data":"a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264"}
Oct 29 23:33:32.663406 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.663325    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth" event={"ID":"6291cdad-e6f2-461f-be7e-d970959ae576","Type":"ContainerDied","Data":"60f9a308fd2b910eef238c222d98406c961902b8a5fc4241c95dc9125ecf7389"}
Oct 29 23:33:32.674849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.674815    6040 scope.go:122] "RemoveContainer" containerID="94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa"
Oct 29 23:33:32.675238 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:33:32.675206    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa\": container with ID starting with 94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa not found: ID does not exist" containerID="94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa"
Oct 29 23:33:32.675370 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.675234    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa"} err="failed to get container status \"94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa\": rpc error: code = NotFound desc = could not find container \"94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa\": container with ID starting with 94e5c066082b158161400bdbbef066c33a02211393963d64954b91be80093dfa not found: ID does not exist"
Oct 29 23:33:32.675370 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.675255    6040 scope.go:122] "RemoveContainer" containerID="a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264"
Oct 29 23:33:32.687002 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.686973    6040 scope.go:122] "RemoveContainer" containerID="a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264"
Oct 29 23:33:32.687382 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:33:32.687355    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264\": container with ID starting with a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264 not found: ID does not exist" containerID="a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264"
Oct 29 23:33:32.687496 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.687394    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264"} err="failed to get container status \"a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264\": rpc error: code = NotFound desc = could not find container \"a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264\": container with ID starting with a6d68d749bfe728d7f0552ff9bcc26969925a952ff5457fc19d4b3bf00aa8264 not found: ID does not exist"
Oct 29 23:33:32.717732 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.717697    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"]
Oct 29 23:33:32.724257 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.724225    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-lxj5v"]
Oct 29 23:33:32.763187 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.763153    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"]
Oct 29 23:33:32.769826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.769715    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-t6nth"]
Oct 29 23:33:32.943724 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.943694    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:32.955062 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:32.955032    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:33.244732 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.244684    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main" containerID="cri-o://b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73" gracePeriod=45
Oct 29 23:33:33.509996 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.509936    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main" containerID="cri-o://d1c2cf97632ca058938093818af923436cad5188ce0d09b0bdccc2dbc911ad85" gracePeriod=44
Oct 29 23:33:33.585829 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.585800    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:33:33.696393 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.696351    6040 generic.go:356] "Generic (PLEG): container finished" podID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerID="b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73" exitCode=0
Oct 29 23:33:33.696602 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.696423    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"
Oct 29 23:33:33.696602 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.696433    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" event={"ID":"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0","Type":"ContainerDied","Data":"b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73"}
Oct 29 23:33:33.696602 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.696485    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n" event={"ID":"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0","Type":"ContainerDied","Data":"1a8c9078e9b822c4b5f6160d0c72a4f96b8bf59b48c00177c49b11cf1b2c85fa"}
Oct 29 23:33:33.696602 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.696509    6040 scope.go:122] "RemoveContainer" containerID="b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73"
Oct 29 23:33:33.698819 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.698792    6040 generic.go:356] "Generic (PLEG): container finished" podID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerID="d1c2cf97632ca058938093818af923436cad5188ce0d09b0bdccc2dbc911ad85" exitCode=0
Oct 29 23:33:33.699209 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.698879    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" event={"ID":"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d","Type":"ContainerDied","Data":"d1c2cf97632ca058938093818af923436cad5188ce0d09b0bdccc2dbc911ad85"}
Oct 29 23:33:33.708440 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.708410    6040 scope.go:122] "RemoveContainer" containerID="b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73"
Oct 29 23:33:33.708846 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:33:33.708823    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73\": container with ID starting with b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73 not found: ID does not exist" containerID="b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73"
Oct 29 23:33:33.708964 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.708853    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73"} err="failed to get container status \"b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73\": rpc error: code = NotFound desc = could not find container \"b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73\": container with ID starting with b971ef77791ec4365ab41e0c1e1037ad09d1ba8b596f85f27b245337a2c3bc73 not found: ID does not exist"
Oct 29 23:33:33.729355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.729318    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-model-cache\") pod \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") "
Oct 29 23:33:33.729500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.729375    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-dshm\") pod \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") "
Oct 29 23:33:33.729500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.729458    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tls-certs\" (UniqueName: \"kubernetes.io/secret/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tls-certs\") pod \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") "
Oct 29 23:33:33.729578 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.729504    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-home\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-home\") pod \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") "
Oct 29 23:33:33.729578 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.729543    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tmp-dir\") pod \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") "
Oct 29 23:33:33.729711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.729693    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-kube-api-access-4ts8d\" (UniqueName: \"kubernetes.io/projected/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-kube-api-access-4ts8d\") pod \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\" (UID: \"0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0\") "
Oct 29 23:33:33.733579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.733541    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-kube-api-access-4ts8d" pod "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" (UID: "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0"). InnerVolumeSpecName "kube-api-access-4ts8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:33.733825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.733800    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-model-cache" pod "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" (UID: "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:33.734021 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.734000    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-home" pod "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" (UID: "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:33.734466 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.734438    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tmp-dir" pod "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" (UID: "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:33.736544 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.736516    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-dshm" pod "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" (UID: "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:33.737391 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.737372    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tls-certs" pod "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" (UID: "0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:33:33.831439 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.831408    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:33.831439 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.831436    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:33.831635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.831448    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:33.831635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.831460    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ts8d\" (UniqueName: \"kubernetes.io/projected/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-kube-api-access-4ts8d\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:33.831635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.831472    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:33.831635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.831484    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:33.832058 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.832034    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:33:33.932457 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.932424    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-dshm\") pod \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") "
Oct 29 23:33:33.932617 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.932530    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-model-cache\") pod \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") "
Oct 29 23:33:33.932617 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.932590    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-home\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-home\") pod \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") "
Oct 29 23:33:33.932773 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.932626    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tls-certs\" (UniqueName: \"kubernetes.io/secret/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tls-certs\") pod \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") "
Oct 29 23:33:33.932773 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.932657    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-kube-api-access-cjbdm\" (UniqueName: \"kubernetes.io/projected/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-kube-api-access-cjbdm\") pod \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") "
Oct 29 23:33:33.932846 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.932821    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tmp-dir\") pod \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\" (UID: \"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d\") "
Oct 29 23:33:33.932894 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.932851    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-model-cache" pod "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" (UID: "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:33.932983 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.932954    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-home" pod "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" (UID: "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:33.933275 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.933251    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tmp-dir" pod "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" (UID: "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:33.933476 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.933439    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:33.933476 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.933461    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:33.933476 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.933475    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:33.939906 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.935064    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tls-certs" pod "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" (UID: "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:33:33.939906 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.935194    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-dshm" pod "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" (UID: "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:33.939906 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:33.935250    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-kube-api-access-cjbdm" pod "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" (UID: "658cbe09-1fd9-4438-a5ba-ef3d98bfe46d"). InnerVolumeSpecName "kube-api-access-cjbdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:34.035239 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.035204    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:34.035239 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.035235    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjbdm\" (UniqueName: \"kubernetes.io/projected/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-kube-api-access-cjbdm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:34.035239 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.035247    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:34.045694 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.044918    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"]
Oct 29 23:33:34.048817 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.048789    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/distinct-2-w0-b978a0-kserve-7dbf58f4d5-24p5n"]
Oct 29 23:33:34.426751 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.426701    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" path="/var/lib/kubelet/pods/0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0/volumes"
Oct 29 23:33:34.427448 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.427408    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6291cdad-e6f2-461f-be7e-d970959ae576" path="/var/lib/kubelet/pods/6291cdad-e6f2-461f-be7e-d970959ae576/volumes"
Oct 29 23:33:34.428018 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.428001    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="751a2a59-c2db-498e-8d07-eba400912132" path="/var/lib/kubelet/pods/751a2a59-c2db-498e-8d07-eba400912132/volumes"
Oct 29 23:33:34.710960 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.710870    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"
Oct 29 23:33:34.710960 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.710913    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd" event={"ID":"658cbe09-1fd9-4438-a5ba-ef3d98bfe46d","Type":"ContainerDied","Data":"9dd1d2a7cefac7f531eba7f0bb0295890c9f0bd9874db77ab50aa5c1fd2f6c0c"}
Oct 29 23:33:34.711179 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.710966    6040 scope.go:122] "RemoveContainer" containerID="d1c2cf97632ca058938093818af923436cad5188ce0d09b0bdccc2dbc911ad85"
Oct 29 23:33:34.763343 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.763307    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"]
Oct 29 23:33:34.767357 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:34.767331    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-b978a0/premium-w0-b978a0-kserve-5c845494dc-pnfdd"]
Oct 29 23:33:36.428564 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:36.428531    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" path="/var/lib/kubelet/pods/658cbe09-1fd9-4438-a5ba-ef3d98bfe46d/volumes"
Oct 29 23:33:38.487885 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:38.487847    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"]
Oct 29 23:33:38.488419 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:38.488106    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy" containerID="cri-o://5372e7a0d1fa99c442c2f2bfc7b792b459272039737483cab609051f3280aa40" gracePeriod=30
Oct 29 23:33:38.505545 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:38.505511    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"]
Oct 29 23:33:38.505996 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:38.505963    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952" podUID="ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" containerName="istio-proxy" containerID="cri-o://a944855ff49c57664a190b8a4324e2cc8e65fbfce29ec81cedf163fafeb42685" gracePeriod=30
Oct 29 23:33:38.510626 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:38.510562    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"]
Oct 29 23:33:38.510803 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:38.510779    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy" containerID="cri-o://463f241c32fdfa045a165e30e11ac524ac21c6bf4b5cc2946b608c1f3ff77a59" gracePeriod=30
Oct 29 23:33:39.767365 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.767331    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:33:39.781561 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.781523    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"]
Oct 29 23:33:39.782345 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782320    6040 cpu_manager.go:414] "containerMap: removing container" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main"
Oct 29 23:33:39.782345 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782346    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main"
Oct 29 23:33:39.782516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782364    6040 cpu_manager.go:414] "containerMap: removing container" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerName="istio-proxy"
Oct 29 23:33:39.782516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782372    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerName="istio-proxy"
Oct 29 23:33:39.782516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782414    6040 cpu_manager.go:414] "containerMap: removing container" podUID="751a2a59-c2db-498e-8d07-eba400912132" containerName="istio-proxy"
Oct 29 23:33:39.782516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782423    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="751a2a59-c2db-498e-8d07-eba400912132" containerName="istio-proxy"
Oct 29 23:33:39.782516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782436    6040 cpu_manager.go:414] "containerMap: removing container" podUID="6291cdad-e6f2-461f-be7e-d970959ae576" containerName="istio-proxy"
Oct 29 23:33:39.782516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782443    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6291cdad-e6f2-461f-be7e-d970959ae576" containerName="istio-proxy"
Oct 29 23:33:39.782516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782463    6040 cpu_manager.go:414] "containerMap: removing container" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main"
Oct 29 23:33:39.782516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782470    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main"
Oct 29 23:33:39.782962 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782599    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="751a2a59-c2db-498e-8d07-eba400912132" containerName="istio-proxy"
Oct 29 23:33:39.782962 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782612    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="6291cdad-e6f2-461f-be7e-d970959ae576" containerName="istio-proxy"
Oct 29 23:33:39.782962 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782624    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="658cbe09-1fd9-4438-a5ba-ef3d98bfe46d" containerName="main"
Oct 29 23:33:39.782962 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782637    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="0bd477c5-2f9a-46c8-a6d6-078f2ec7bbf0" containerName="main"
Oct 29 23:33:39.782962 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.782647    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="696b829d-a17c-48d8-9cbc-ec567e7add16" containerName="istio-proxy"
Oct 29 23:33:39.788139 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.788115    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.817205 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.817167    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"]
Oct 29 23:33:39.820170 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.820133    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.820341 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.820201    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.820341 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.820240    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.820341 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.820264    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4a1ac1b-341e-45da-9958-211394cd4483-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.820341 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.820314    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.820557 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.820354    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4a1ac1b-341e-45da-9958-211394cd4483-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.820557 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.820429    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk2hw\" (UniqueName: \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-kube-api-access-hk2hw\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.820557 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.820518    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.820700 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.820641    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.921504 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.921473    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.921655 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.921518    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.921655 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.921548    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.921655 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.921566    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4a1ac1b-341e-45da-9958-211394cd4483-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.921655 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.921598    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.921655 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.921625    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4a1ac1b-341e-45da-9958-211394cd4483-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.921983 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.921679    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk2hw\" (UniqueName: \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-kube-api-access-hk2hw\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.921983 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.921770    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.921983 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.921886    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.921983 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.921965    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.922111 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.921986    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.922294 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.922268    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.922446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.922422    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.922521 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.922495    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4a1ac1b-341e-45da-9958-211394cd4483-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.923991 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.923966    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.924166 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.924148    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4a1ac1b-341e-45da-9958-211394cd4483-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.945381 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.945354    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:39.945553 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:39.945532    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk2hw\" (UniqueName: \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-kube-api-access-hk2hw\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:40.105029 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:40.104944    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:40.451629 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:33:40.451595    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4a1ac1b_341e_45da_9958_211394cd4483.slice/crio-9b26cc76d96d25990dd23a2d28e5b5feb4a476d770b052b5deb94132a7a25040 WatchSource:0}: Error finding container 9b26cc76d96d25990dd23a2d28e5b5feb4a476d770b052b5deb94132a7a25040: Status 404 returned error can't find the container with id 9b26cc76d96d25990dd23a2d28e5b5feb4a476d770b052b5deb94132a7a25040
Oct 29 23:33:40.454044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:40.454019    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"]
Oct 29 23:33:40.455956 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:40.455923    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:40.456031 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:40.455987    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:40.456076 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:40.456027    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:33:40.810253 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:40.808809    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" event={"ID":"c4a1ac1b-341e-45da-9958-211394cd4483","Type":"ContainerStarted","Data":"e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64"}
Oct 29 23:33:40.810253 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:40.808878    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" event={"ID":"c4a1ac1b-341e-45da-9958-211394cd4483","Type":"ContainerStarted","Data":"9b26cc76d96d25990dd23a2d28e5b5feb4a476d770b052b5deb94132a7a25040"}
Oct 29 23:33:40.856463 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:40.856377    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podStartSLOduration=1.856356861 podStartE2EDuration="1.856356861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:40.855165445 +0000 UTC m=+1200.550274326" watchObservedRunningTime="2026-10-29 23:33:40.856356861 +0000 UTC m=+1200.551465742"
Oct 29 23:33:41.109164 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:41.109079    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:41.112901 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:41.112868    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 29 23:33:41.113149 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:41.113124    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 29 23:33:42.109200 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:42.109160    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 29 23:33:42.110671 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:42.110634    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 29 23:33:43.110419 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:43.110380    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 29 23:33:43.111030 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:43.110994    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 29 23:33:43.122409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:43.122372    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:43.406761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:43.395837    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:43.733918 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:43.733851    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:44.117763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.114861    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 29 23:33:44.117763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.114935    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 29 23:33:44.140809 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.126820    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:44.140809 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.128129    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:33:44.140809 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.139256    6040 generic.go:356] "Generic (PLEG): container finished" podID="ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" containerID="a944855ff49c57664a190b8a4324e2cc8e65fbfce29ec81cedf163fafeb42685" exitCode=0
Oct 29 23:33:44.140809 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.139357    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952" event={"ID":"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2","Type":"ContainerDied","Data":"a944855ff49c57664a190b8a4324e2cc8e65fbfce29ec81cedf163fafeb42685"}
Oct 29 23:33:44.141897 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.141873    6040 generic.go:356] "Generic (PLEG): container finished" podID="be781867-f314-44b4-8899-e66412a92d3d" containerID="463f241c32fdfa045a165e30e11ac524ac21c6bf4b5cc2946b608c1f3ff77a59" exitCode=0
Oct 29 23:33:44.142135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.142114    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" event={"ID":"be781867-f314-44b4-8899-e66412a92d3d","Type":"ContainerDied","Data":"463f241c32fdfa045a165e30e11ac524ac21c6bf4b5cc2946b608c1f3ff77a59"}
Oct 29 23:33:44.161981 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.161771    6040 generic.go:356] "Generic (PLEG): container finished" podID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerID="5372e7a0d1fa99c442c2f2bfc7b792b459272039737483cab609051f3280aa40" exitCode=0
Oct 29 23:33:44.161981 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.161917    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" event={"ID":"be7691e1-6b13-4179-a5f5-5a7af04c6525","Type":"ContainerDied","Data":"5372e7a0d1fa99c442c2f2bfc7b792b459272039737483cab609051f3280aa40"}
Oct 29 23:33:44.396883 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.396834    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:44.453767 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.404845    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:33:44.688441 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.673977    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:44.720762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.690756    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" podUID="07800096-1dc8-41ca-9a09-756653bbf742" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/ready\": context deadline exceeded"
Oct 29 23:33:44.747763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.742928    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:44.884953 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:44.884906    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:45.025070 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.012712    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:33:45.025070 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.013002    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" podUID="680505f8-f92f-4225-b61f-5b323c23b236" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 29 23:33:45.140270 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.140227    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:45.162395 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.162366    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:33:45.284024 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.283937    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 29 23:33:45.284281 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.284245    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 29 23:33:45.339004 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.335570    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": context deadline exceeded"
Oct 29 23:33:45.406466 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.405438    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:45.589812 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.589486    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:33:45.671264 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.670864    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-podinfo\") pod \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") "
Oct 29 23:33:45.671264 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.670967    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-data\") pod \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") "
Oct 29 23:33:45.671264 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.671021    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-certs\") pod \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") "
Oct 29 23:33:45.671264 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.671077    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-credential-socket\") pod \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") "
Oct 29 23:33:45.671264 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.671200    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-envoy\") pod \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") "
Oct 29 23:33:45.671675 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.671326    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-kube-api-access-zqb7b\" (UniqueName: \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-kube-api-access-zqb7b\") pod \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") "
Oct 29 23:33:45.671675 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.671356    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-token\" (UniqueName: \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-token\") pod \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") "
Oct 29 23:33:45.671675 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.671414    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istiod-ca-cert\") pod \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") "
Oct 29 23:33:45.671675 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.671458    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-socket\") pod \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\" (UID: \"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2\") "
Oct 29 23:33:45.679157 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.678107    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-credential-socket" pod "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" (UID: "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:45.734782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.718192    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-data" pod "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" (UID: "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:45.734782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.718435    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 29 23:33:45.734782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.718702    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" podUID="07800096-1dc8-41ca-9a09-756653bbf742" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:45.734782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.719268    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:33:45.734782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.719324    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:45.734782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.730981    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istiod-ca-cert" pod "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" (UID: "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:33:45.734782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.732002    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-certs" pod "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" (UID: "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:45.797276 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.746088    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:45.797276 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.772188    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-socket" pod "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" (UID: "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:45.797276 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.788137    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:45.797276 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.788169    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:45.797276 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.788185    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:45.797276 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.788200    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:45.797276 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.788216    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:45.797276 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.792299    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:33:45.803446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.803309    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-token" pod "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" (UID: "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:45.803446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.803399    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-kube-api-access-zqb7b" pod "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" (UID: "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2"). InnerVolumeSpecName "kube-api-access-zqb7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:45.803446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.803419    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-podinfo" pod "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" (UID: "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:33:45.843674 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.842270    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:33:45.874160 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.873285    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-envoy" pod "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" (UID: "ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:45.882286 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.881396    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:33:45.889771 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.889726    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:45.889771 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.889771    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:45.889988 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.889789    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqb7b\" (UniqueName: \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-kube-api-access-zqb7b\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:45.889988 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:45.889802    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.024360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.013831    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" podUID="680505f8-f92f-4225-b61f-5b323c23b236" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:33:46.024985 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.024822    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:46.111616 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.082221    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": context deadline exceeded"
Oct 29 23:33:46.113216 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.112669    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 29 23:33:46.113216 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.112727    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 29 23:33:46.386415 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.386388    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:33:46.393219 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.391899    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:33:46.394101 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.394081    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:33:46.462263 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.462235    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"
Oct 29 23:33:46.464783 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.464575    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:33:46.466828 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.466721    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:33:46.482012 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.481977    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952" event={"ID":"ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2","Type":"ContainerDied","Data":"e931b3b2186e2f71ccfba07bd8e42ba281641a12932ceed2969644f44b1a55a7"}
Oct 29 23:33:46.482754 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.482712    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm" event={"ID":"be781867-f314-44b4-8899-e66412a92d3d","Type":"ContainerDied","Data":"e6c8b07e70a4eae250d3b6129020c45df3de47bfcc162743289f69a7f9106a46"}
Oct 29 23:33:46.482949 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.482932    6040 scope.go:122] "RemoveContainer" containerID="a944855ff49c57664a190b8a4324e2cc8e65fbfce29ec81cedf163fafeb42685"
Oct 29 23:33:46.495423 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.495358    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s" event={"ID":"be7691e1-6b13-4179-a5f5-5a7af04c6525","Type":"ContainerDied","Data":"acda1dfa91ff0ab65db624ce7bea7e7812c0119148cc74c73d1e1423c295e5dc"}
Oct 29 23:33:46.495567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.495431    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"
Oct 29 23:33:46.545354 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.545324    6040 scope.go:122] "RemoveContainer" containerID="463f241c32fdfa045a165e30e11ac524ac21c6bf4b5cc2946b608c1f3ff77a59"
Oct 29 23:33:46.558097 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.558070    6040 scope.go:122] "RemoveContainer" containerID="5372e7a0d1fa99c442c2f2bfc7b792b459272039737483cab609051f3280aa40"
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625165    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-data\") pod \"be781867-f314-44b4-8899-e66412a92d3d\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625251    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-socket\") pod \"be781867-f314-44b4-8899-e66412a92d3d\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625312    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be781867-f314-44b4-8899-e66412a92d3d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be781867-f314-44b4-8899-e66412a92d3d-istiod-ca-cert\") pod \"be781867-f314-44b4-8899-e66412a92d3d\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625341    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-socket\") pod \"be7691e1-6b13-4179-a5f5-5a7af04c6525\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625394    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-data\") pod \"be7691e1-6b13-4179-a5f5-5a7af04c6525\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625429    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/be781867-f314-44b4-8899-e66412a92d3d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be781867-f314-44b4-8899-e66412a92d3d-istio-podinfo\") pod \"be781867-f314-44b4-8899-e66412a92d3d\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625471    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-certs\") pod \"be781867-f314-44b4-8899-e66412a92d3d\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625522    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-credential-socket\") pod \"be781867-f314-44b4-8899-e66412a92d3d\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625596    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-token\" (UniqueName: \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-token\") pod \"be7691e1-6b13-4179-a5f5-5a7af04c6525\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625666    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-certs\") pod \"be7691e1-6b13-4179-a5f5-5a7af04c6525\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625699    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-istio-token\" (UniqueName: \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-istio-token\") pod \"be781867-f314-44b4-8899-e66412a92d3d\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625728    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-podinfo\") pod \"be7691e1-6b13-4179-a5f5-5a7af04c6525\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625866    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-credential-socket\") pod \"be7691e1-6b13-4179-a5f5-5a7af04c6525\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625896    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be7691e1-6b13-4179-a5f5-5a7af04c6525-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7691e1-6b13-4179-a5f5-5a7af04c6525-istiod-ca-cert\") pod \"be7691e1-6b13-4179-a5f5-5a7af04c6525\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625936    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-envoy\") pod \"be7691e1-6b13-4179-a5f5-5a7af04c6525\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.625973    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-kube-api-access-w8pvn\" (UniqueName: \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-kube-api-access-w8pvn\") pod \"be7691e1-6b13-4179-a5f5-5a7af04c6525\" (UID: \"be7691e1-6b13-4179-a5f5-5a7af04c6525\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.626042    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-kube-api-access-94cqs\" (UniqueName: \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-kube-api-access-94cqs\") pod \"be781867-f314-44b4-8899-e66412a92d3d\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") "
Oct 29 23:33:46.628892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.626078    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-envoy\") pod \"be781867-f314-44b4-8899-e66412a92d3d\" (UID: \"be781867-f314-44b4-8899-e66412a92d3d\") "
Oct 29 23:33:46.630355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.630321    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-envoy" pod "be781867-f314-44b4-8899-e66412a92d3d" (UID: "be781867-f314-44b4-8899-e66412a92d3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:46.630514 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.630498    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-data" pod "be781867-f314-44b4-8899-e66412a92d3d" (UID: "be781867-f314-44b4-8899-e66412a92d3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:46.630658 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.630644    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-socket" pod "be781867-f314-44b4-8899-e66412a92d3d" (UID: "be781867-f314-44b4-8899-e66412a92d3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:46.632714 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.631375    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-certs" pod "be7691e1-6b13-4179-a5f5-5a7af04c6525" (UID: "be7691e1-6b13-4179-a5f5-5a7af04c6525"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:46.632714 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.631584    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-socket" pod "be7691e1-6b13-4179-a5f5-5a7af04c6525" (UID: "be7691e1-6b13-4179-a5f5-5a7af04c6525"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:46.632714 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.631707    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-certs" pod "be781867-f314-44b4-8899-e66412a92d3d" (UID: "be781867-f314-44b4-8899-e66412a92d3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:46.632714 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.631796    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-credential-socket" pod "be781867-f314-44b4-8899-e66412a92d3d" (UID: "be781867-f314-44b4-8899-e66412a92d3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:46.632714 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.631934    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-data" pod "be7691e1-6b13-4179-a5f5-5a7af04c6525" (UID: "be7691e1-6b13-4179-a5f5-5a7af04c6525"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:46.632714 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.632081    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be781867-f314-44b4-8899-e66412a92d3d-istiod-ca-cert" pod "be781867-f314-44b4-8899-e66412a92d3d" (UID: "be781867-f314-44b4-8899-e66412a92d3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:33:46.632714 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.632238    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be7691e1-6b13-4179-a5f5-5a7af04c6525-istiod-ca-cert" pod "be7691e1-6b13-4179-a5f5-5a7af04c6525" (UID: "be7691e1-6b13-4179-a5f5-5a7af04c6525"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:33:46.634415 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.634384    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"]
Oct 29 23:33:46.634704 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.634679    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-token" pod "be7691e1-6b13-4179-a5f5-5a7af04c6525" (UID: "be7691e1-6b13-4179-a5f5-5a7af04c6525"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:46.634831 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.634757    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-credential-socket" pod "be7691e1-6b13-4179-a5f5-5a7af04c6525" (UID: "be7691e1-6b13-4179-a5f5-5a7af04c6525"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:46.634926 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.634905    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-istio-token" pod "be781867-f314-44b4-8899-e66412a92d3d" (UID: "be781867-f314-44b4-8899-e66412a92d3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:46.635252 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.635226    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be781867-f314-44b4-8899-e66412a92d3d-istio-podinfo" pod "be781867-f314-44b4-8899-e66412a92d3d" (UID: "be781867-f314-44b4-8899-e66412a92d3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:33:46.635391 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.635372    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-podinfo" pod "be7691e1-6b13-4179-a5f5-5a7af04c6525" (UID: "be7691e1-6b13-4179-a5f5-5a7af04c6525"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:33:46.635970 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.635942    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-envoy" pod "be7691e1-6b13-4179-a5f5-5a7af04c6525" (UID: "be7691e1-6b13-4179-a5f5-5a7af04c6525"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:33:46.636429 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.636412    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-kube-api-access-94cqs" pod "be781867-f314-44b4-8899-e66412a92d3d" (UID: "be781867-f314-44b4-8899-e66412a92d3d"). InnerVolumeSpecName "kube-api-access-94cqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:46.637033 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.637018    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-kube-api-access-w8pvn" pod "be7691e1-6b13-4179-a5f5-5a7af04c6525" (UID: "be7691e1-6b13-4179-a5f5-5a7af04c6525"). InnerVolumeSpecName "kube-api-access-w8pvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:33:46.637621 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.637600    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-jn952"]
Oct 29 23:33:46.732318 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732278    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.732537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732520    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be781867-f314-44b4-8899-e66412a92d3d-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.732640 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732626    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.732791 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732775    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.732907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732819    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.732907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732838    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.732907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732851    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.732907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732864    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.732907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732876    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.732907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732889    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7691e1-6b13-4179-a5f5-5a7af04c6525-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.732907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732902    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.732907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732916    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8pvn\" (UniqueName: \"kubernetes.io/projected/be7691e1-6b13-4179-a5f5-5a7af04c6525-kube-api-access-w8pvn\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.733236 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732930    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94cqs\" (UniqueName: \"kubernetes.io/projected/be781867-f314-44b4-8899-e66412a92d3d-kube-api-access-94cqs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.733236 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732943    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.733236 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732956    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.733236 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732971    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be781867-f314-44b4-8899-e66412a92d3d-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.733236 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732984    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be781867-f314-44b4-8899-e66412a92d3d-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.733236 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.732997    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7691e1-6b13-4179-a5f5-5a7af04c6525-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:33:46.873594 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.873566    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"]
Oct 29 23:33:46.880115 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:46.880049    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-k8p8s"]
Oct 29 23:33:47.126178 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:47.126142    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 29 23:33:47.126416 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:47.126207    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 29 23:33:47.523759 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:47.521928    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"
Oct 29 23:33:47.579168 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:47.579088    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"]
Oct 29 23:33:47.590648 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:47.590618    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-7lhjm"]
Oct 29 23:33:48.126991 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:48.126140    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 29 23:33:48.126991 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:48.126199    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 29 23:33:48.439770 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:48.439701    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" path="/var/lib/kubelet/pods/be7691e1-6b13-4179-a5f5-5a7af04c6525/volumes"
Oct 29 23:33:48.440228 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:48.440212    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be781867-f314-44b4-8899-e66412a92d3d" path="/var/lib/kubelet/pods/be781867-f314-44b4-8899-e66412a92d3d/volumes"
Oct 29 23:33:48.440795 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:48.440776    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" path="/var/lib/kubelet/pods/ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2/volumes"
Oct 29 23:33:49.121762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:49.121443    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 29 23:33:49.121762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:49.121506    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 29 23:33:50.105459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:50.105425    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 29 23:33:50.105644 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:50.105491    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 29 23:33:51.110373 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:51.110335    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:51.110767 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:51.110610    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:51.111244 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:51.111223    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:33:58.186984 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.186936    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"]
Oct 29 23:33:58.187807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.187788    6040 cpu_manager.go:414] "containerMap: removing container" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy"
Oct 29 23:33:58.187922 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.187808    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy"
Oct 29 23:33:58.187922 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.187832    6040 cpu_manager.go:414] "containerMap: removing container" podUID="ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" containerName="istio-proxy"
Oct 29 23:33:58.187922 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.187839    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" containerName="istio-proxy"
Oct 29 23:33:58.187922 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.187858    6040 cpu_manager.go:414] "containerMap: removing container" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy"
Oct 29 23:33:58.187922 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.187866    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy"
Oct 29 23:33:58.188174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.188031    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="be781867-f314-44b4-8899-e66412a92d3d" containerName="istio-proxy"
Oct 29 23:33:58.188174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.188057    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="be7691e1-6b13-4179-a5f5-5a7af04c6525" containerName="istio-proxy"
Oct 29 23:33:58.188174 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.188066    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebd5e5a9-6c82-453e-9be5-dfc9df17a6e2" containerName="istio-proxy"
Oct 29 23:33:58.191502 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.191483    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.209413 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.209387    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"]
Oct 29 23:33:58.221355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.217565    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-aa7e96\"/\"distinct-2-w4-aa7e96-kserve-self-signed-certs\""
Oct 29 23:33:58.280108 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.280069    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-dshm\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.280283 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.280179    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5qwz\" (UniqueName: \"kubernetes.io/projected/186078cc-b997-4651-9dcf-b33603f13047-kube-api-access-w5qwz\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.280283 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.280242    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-model-cache\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.280283 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.280279    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/186078cc-b997-4651-9dcf-b33603f13047-tls-certs\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.280414 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.280385    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-tmp-dir\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.280532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.280515    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-home\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.381540 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.381448    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-dshm\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.382272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.382248    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5qwz\" (UniqueName: \"kubernetes.io/projected/186078cc-b997-4651-9dcf-b33603f13047-kube-api-access-w5qwz\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.382496 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.382477    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-model-cache\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.382649 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.382633    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/186078cc-b997-4651-9dcf-b33603f13047-tls-certs\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.382826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.382811    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-tmp-dir\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.383096 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.383079    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-home\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.383612 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.383591    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-home\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.389775 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.385954    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-tmp-dir\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.390659 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.390630    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/186078cc-b997-4651-9dcf-b33603f13047-tls-certs\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.403833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.403795    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-model-cache\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.416224 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.416185    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-dshm\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.417114 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.417080    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5qwz\" (UniqueName: \"kubernetes.io/projected/186078cc-b997-4651-9dcf-b33603f13047-kube-api-access-w5qwz\") pod \"distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") " pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.505187 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.505093    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:33:58.854869 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:58.854836    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"]
Oct 29 23:33:58.855466 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:33:58.855438    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod186078cc_b997_4651_9dcf_b33603f13047.slice/crio-7e24f2fb28429bccea377b3e2a12650385c5df872b6547f8121fe4a3d4479371 WatchSource:0}: Error finding container 7e24f2fb28429bccea377b3e2a12650385c5df872b6547f8121fe4a3d4479371: Status 404 returned error can't find the container with id 7e24f2fb28429bccea377b3e2a12650385c5df872b6547f8121fe4a3d4479371
Oct 29 23:33:59.621042 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:59.621006    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t" event={"ID":"186078cc-b997-4651-9dcf-b33603f13047","Type":"ContainerStarted","Data":"6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c"}
Oct 29 23:33:59.621530 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:59.621051    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t" event={"ID":"186078cc-b997-4651-9dcf-b33603f13047","Type":"ContainerStarted","Data":"7e24f2fb28429bccea377b3e2a12650385c5df872b6547f8121fe4a3d4479371"}
Oct 29 23:33:59.666551 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:33:59.666482    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t" podStartSLOduration=1.6664624959999998 podStartE2EDuration="1.666462496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:59.661985576 +0000 UTC m=+1219.357094455" watchObservedRunningTime="2026-10-29 23:33:59.666462496 +0000 UTC m=+1219.361571379"
Oct 29 23:33:59.784284 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:33:59.784239    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:34:00.506041 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:00.506009    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:34:05.814067 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:05.814034    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"]
Oct 29 23:34:05.814623 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:05.814286    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerName="istio-proxy" containerID="cri-o://3bc838ab49f5e603fb7f88a45c37be2291e5e047ca91651a5a81a7336219dc74" gracePeriod=30
Oct 29 23:34:05.847904 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:05.847867    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"]
Oct 29 23:34:05.848122 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:05.848101    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" podUID="9fd7b03f-a363-415c-9397-5d202774916d" containerName="istio-proxy" containerID="cri-o://4daa2be0fbcf31fb559427af6f300a038f84b66e5c8d3d5f4a19091c6ae4dafd" gracePeriod=30
Oct 29 23:34:05.851216 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:05.851189    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"]
Oct 29 23:34:05.851455 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:05.851432    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" podUID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" containerName="istio-proxy" containerID="cri-o://e8e95ceaff603a3321ed7649b720b89f79c599b719da41e6af30812bcf23367c" gracePeriod=30
Oct 29 23:34:06.896722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:06.896686    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:34:08.508760 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:08.508517    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:34:09.408894 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:09.408842    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" podUID="680505f8-f92f-4225-b61f-5b323c23b236" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:09.410425 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:09.410394    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:09.410962 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:09.410937    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 29 23:34:09.508763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:09.507831    6040 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t" podUID="186078cc-b997-4651-9dcf-b33603f13047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:09.752823 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:09.751954    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:09.794990 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:09.794947    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:09.850279 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:09.850227    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" podUID="07800096-1dc8-41ca-9a09-756653bbf742" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:09.885895 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:09.885848    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:09.982927 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:09.982864    6040 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:10.170774 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.166875    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:10.413761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.409838    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" podUID="680505f8-f92f-4225-b61f-5b323c23b236" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:10.422391 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.414117    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:10.453946 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.423915    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:34:10.454263 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.454248    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:34:10.510807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.424945    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:10.578020 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.577987    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:34:10.872521 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.796687    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:34:10.872521 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.792893    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-slh2h" podUID="bf347111-37c9-481c-b4b1-a59fb1bf3422" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.133.0.19:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:10.872521 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.803657    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 29 23:34:10.872521 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.848491    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:34:10.872521 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.848557    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:34:10.910238 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.890189    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:10.910238 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.890628    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:34:10.910238 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.891967    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" podUID="07800096-1dc8-41ca-9a09-756653bbf742" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/ready\": context deadline exceeded"
Oct 29 23:34:10.910238 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:10.892274    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:34:11.361634 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:11.361582    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:34:11.362305 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:11.362284    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:34:11.395517 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:11.395479    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:34:11.428335 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:11.408853    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:34:11.428335 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:11.409560    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:34:11.428335 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:11.421193    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" podUID="680505f8-f92f-4225-b61f-5b323c23b236" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 29 23:34:11.452506 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:11.450863    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:11.981217 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:11.981171    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" podUID="07800096-1dc8-41ca-9a09-756653bbf742" containerName="main" probeResult="failure" output="Get \"https://10.133.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:11.991129 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:11.991088    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:34:12.217481 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.217379    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:34:12.290995 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.290950    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:34:12.291265 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.291247    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:34:12.293499 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.293468    6040 generic.go:356] "Generic (PLEG): container finished" podID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerID="3bc838ab49f5e603fb7f88a45c37be2291e5e047ca91651a5a81a7336219dc74" exitCode=0
Oct 29 23:34:12.293837 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.293812    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" event={"ID":"be7a7c5a-d8b8-4f9d-86f2-435dc7146711","Type":"ContainerDied","Data":"3bc838ab49f5e603fb7f88a45c37be2291e5e047ca91651a5a81a7336219dc74"}
Oct 29 23:34:12.364787 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.362351    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 29 23:34:12.364787 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.362645    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:34:12.365725 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.365695    6040 generic.go:356] "Generic (PLEG): container finished" podID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" containerID="e8e95ceaff603a3321ed7649b720b89f79c599b719da41e6af30812bcf23367c" exitCode=0
Oct 29 23:34:12.366077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.365827    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" event={"ID":"42d13594-12eb-4f2c-8ce7-6c92cfec635d","Type":"ContainerDied","Data":"e8e95ceaff603a3321ed7649b720b89f79c599b719da41e6af30812bcf23367c"}
Oct 29 23:34:12.371561 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.370140    6040 generic.go:356] "Generic (PLEG): container finished" podID="9fd7b03f-a363-415c-9397-5d202774916d" containerID="4daa2be0fbcf31fb559427af6f300a038f84b66e5c8d3d5f4a19091c6ae4dafd" exitCode=0
Oct 29 23:34:12.371561 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.371220    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" event={"ID":"9fd7b03f-a363-415c-9397-5d202774916d","Type":"ContainerDied","Data":"4daa2be0fbcf31fb559427af6f300a038f84b66e5c8d3d5f4a19091c6ae4dafd"}
Oct 29 23:34:12.385219 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.385190    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:34:12.398437 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.398389    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:34:12.400526 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.400491    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:34:12.658774 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.658711    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:34:12.696059 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.696032    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:34:12.712217 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.712188    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:34:12.756038 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756006    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-certs\") pod \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") "
Oct 29 23:34:12.756231 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756097    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-data\") pod \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") "
Oct 29 23:34:12.756231 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756132    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-token\" (UniqueName: \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-token\") pod \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") "
Oct 29 23:34:12.756231 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756165    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-credential-socket\") pod \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") "
Oct 29 23:34:12.756231 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756198    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-kube-api-access-tchsm\" (UniqueName: \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-kube-api-access-tchsm\") pod \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") "
Oct 29 23:34:12.756511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756283    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-envoy\") pod \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") "
Oct 29 23:34:12.756511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756333    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-certs" pod "be7a7c5a-d8b8-4f9d-86f2-435dc7146711" (UID: "be7a7c5a-d8b8-4f9d-86f2-435dc7146711"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.756511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756391    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-socket\") pod \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") "
Oct 29 23:34:12.756511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756423    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-podinfo\") pod \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") "
Oct 29 23:34:12.756511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756454    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istiod-ca-cert\") pod \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\" (UID: \"be7a7c5a-d8b8-4f9d-86f2-435dc7146711\") "
Oct 29 23:34:12.756802 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756386    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-data" pod "be7a7c5a-d8b8-4f9d-86f2-435dc7146711" (UID: "be7a7c5a-d8b8-4f9d-86f2-435dc7146711"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.756802 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756582    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-credential-socket" pod "be7a7c5a-d8b8-4f9d-86f2-435dc7146711" (UID: "be7a7c5a-d8b8-4f9d-86f2-435dc7146711"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.756802 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.756760    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-socket" pod "be7a7c5a-d8b8-4f9d-86f2-435dc7146711" (UID: "be7a7c5a-d8b8-4f9d-86f2-435dc7146711"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.757082 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.757060    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istiod-ca-cert" pod "be7a7c5a-d8b8-4f9d-86f2-435dc7146711" (UID: "be7a7c5a-d8b8-4f9d-86f2-435dc7146711"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:34:12.757232 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.757212    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.757297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.757235    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.757297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.757249    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.757297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.757262    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.757297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.757275    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.759108 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.759056    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-kube-api-access-tchsm" pod "be7a7c5a-d8b8-4f9d-86f2-435dc7146711" (UID: "be7a7c5a-d8b8-4f9d-86f2-435dc7146711"). InnerVolumeSpecName "kube-api-access-tchsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:34:12.759279 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.759125    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-token" pod "be7a7c5a-d8b8-4f9d-86f2-435dc7146711" (UID: "be7a7c5a-d8b8-4f9d-86f2-435dc7146711"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:34:12.759552 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.759522    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-envoy" pod "be7a7c5a-d8b8-4f9d-86f2-435dc7146711" (UID: "be7a7c5a-d8b8-4f9d-86f2-435dc7146711"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.759649 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.759552    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-podinfo" pod "be7a7c5a-d8b8-4f9d-86f2-435dc7146711" (UID: "be7a7c5a-d8b8-4f9d-86f2-435dc7146711"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:34:12.858500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858356    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-kube-api-access-v6df2\" (UniqueName: \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-kube-api-access-v6df2\") pod \"9fd7b03f-a363-415c-9397-5d202774916d\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") "
Oct 29 23:34:12.858849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858498    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-credential-socket\") pod \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") "
Oct 29 23:34:12.858849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858568    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-certs\") pod \"9fd7b03f-a363-415c-9397-5d202774916d\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") "
Oct 29 23:34:12.858849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858629    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-credential-socket\") pod \"9fd7b03f-a363-415c-9397-5d202774916d\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") "
Oct 29 23:34:12.858849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858655    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-data\") pod \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") "
Oct 29 23:34:12.858849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858691    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-envoy\") pod \"9fd7b03f-a363-415c-9397-5d202774916d\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") "
Oct 29 23:34:12.858849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858725    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-socket\") pod \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") "
Oct 29 23:34:12.858849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858806    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-socket\") pod \"9fd7b03f-a363-415c-9397-5d202774916d\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") "
Oct 29 23:34:12.859275 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858834    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-credential-socket" pod "42d13594-12eb-4f2c-8ce7-6c92cfec635d" (UID: "42d13594-12eb-4f2c-8ce7-6c92cfec635d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.859275 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858854    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9fd7b03f-a363-415c-9397-5d202774916d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd7b03f-a363-415c-9397-5d202774916d-istio-podinfo\") pod \"9fd7b03f-a363-415c-9397-5d202774916d\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") "
Oct 29 23:34:12.859275 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858933    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-data\") pod \"9fd7b03f-a363-415c-9397-5d202774916d\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") "
Oct 29 23:34:12.859275 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.858986    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-certs\") pod \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") "
Oct 29 23:34:12.859275 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.859084    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-podinfo\") pod \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") "
Oct 29 23:34:12.859275 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.859120    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-token\" (UniqueName: \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-token\") pod \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") "
Oct 29 23:34:12.859275 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.859150    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-envoy\") pod \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") "
Oct 29 23:34:12.859275 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.859213    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fd7b03f-a363-415c-9397-5d202774916d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd7b03f-a363-415c-9397-5d202774916d-istiod-ca-cert\") pod \"9fd7b03f-a363-415c-9397-5d202774916d\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") "
Oct 29 23:34:12.859275 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.859249    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-istio-token\" (UniqueName: \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-istio-token\") pod \"9fd7b03f-a363-415c-9397-5d202774916d\" (UID: \"9fd7b03f-a363-415c-9397-5d202774916d\") "
Oct 29 23:34:12.859710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.859299    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istiod-ca-cert\") pod \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") "
Oct 29 23:34:12.859710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.859330    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-kube-api-access-6f5qg\" (UniqueName: \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-kube-api-access-6f5qg\") pod \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\" (UID: \"42d13594-12eb-4f2c-8ce7-6c92cfec635d\") "
Oct 29 23:34:12.860121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.860014    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.860121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.860039    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.860121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.860053    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tchsm\" (UniqueName: \"kubernetes.io/projected/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-kube-api-access-tchsm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.860121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.860069    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.860121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.860066    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-credential-socket" pod "9fd7b03f-a363-415c-9397-5d202774916d" (UID: "9fd7b03f-a363-415c-9397-5d202774916d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.860121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.860083    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7a7c5a-d8b8-4f9d-86f2-435dc7146711-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.860546 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.860227    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-certs" pod "9fd7b03f-a363-415c-9397-5d202774916d" (UID: "9fd7b03f-a363-415c-9397-5d202774916d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.860546 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.860477    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-data" pod "42d13594-12eb-4f2c-8ce7-6c92cfec635d" (UID: "42d13594-12eb-4f2c-8ce7-6c92cfec635d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.860663 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.860618    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-socket" pod "9fd7b03f-a363-415c-9397-5d202774916d" (UID: "9fd7b03f-a363-415c-9397-5d202774916d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.860912 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.860889    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-certs" pod "42d13594-12eb-4f2c-8ce7-6c92cfec635d" (UID: "42d13594-12eb-4f2c-8ce7-6c92cfec635d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.860993 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.860916    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fd7b03f-a363-415c-9397-5d202774916d-istiod-ca-cert" pod "9fd7b03f-a363-415c-9397-5d202774916d" (UID: "9fd7b03f-a363-415c-9397-5d202774916d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:34:12.861129 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.861108    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-data" pod "9fd7b03f-a363-415c-9397-5d202774916d" (UID: "9fd7b03f-a363-415c-9397-5d202774916d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.861767 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.861291    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fd7b03f-a363-415c-9397-5d202774916d-istio-podinfo" pod "9fd7b03f-a363-415c-9397-5d202774916d" (UID: "9fd7b03f-a363-415c-9397-5d202774916d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:34:12.861767 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.861596    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istiod-ca-cert" pod "42d13594-12eb-4f2c-8ce7-6c92cfec635d" (UID: "42d13594-12eb-4f2c-8ce7-6c92cfec635d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:34:12.861767 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.861692    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-kube-api-access-v6df2" pod "9fd7b03f-a363-415c-9397-5d202774916d" (UID: "9fd7b03f-a363-415c-9397-5d202774916d"). InnerVolumeSpecName "kube-api-access-v6df2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:34:12.862027 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.862004    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-socket" pod "42d13594-12eb-4f2c-8ce7-6c92cfec635d" (UID: "42d13594-12eb-4f2c-8ce7-6c92cfec635d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.863060 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.862980    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-envoy" pod "42d13594-12eb-4f2c-8ce7-6c92cfec635d" (UID: "42d13594-12eb-4f2c-8ce7-6c92cfec635d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.865367 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.865342    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-istio-token" pod "9fd7b03f-a363-415c-9397-5d202774916d" (UID: "9fd7b03f-a363-415c-9397-5d202774916d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:34:12.865583 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.865552    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-kube-api-access-6f5qg" pod "42d13594-12eb-4f2c-8ce7-6c92cfec635d" (UID: "42d13594-12eb-4f2c-8ce7-6c92cfec635d"). InnerVolumeSpecName "kube-api-access-6f5qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:34:12.865651 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.865606    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-podinfo" pod "42d13594-12eb-4f2c-8ce7-6c92cfec635d" (UID: "42d13594-12eb-4f2c-8ce7-6c92cfec635d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:34:12.865825 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.865806    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-token" pod "42d13594-12eb-4f2c-8ce7-6c92cfec635d" (UID: "42d13594-12eb-4f2c-8ce7-6c92cfec635d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:34:12.866009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.865994    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-envoy" pod "9fd7b03f-a363-415c-9397-5d202774916d" (UID: "9fd7b03f-a363-415c-9397-5d202774916d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:12.961532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961500    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6df2\" (UniqueName: \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-kube-api-access-v6df2\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961531    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961540    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961549    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961557    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961565    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961572    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961581    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd7b03f-a363-415c-9397-5d202774916d-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961590    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd7b03f-a363-415c-9397-5d202774916d-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961598    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961605    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961612    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961619    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961628    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd7b03f-a363-415c-9397-5d202774916d-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961635    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd7b03f-a363-415c-9397-5d202774916d-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961642    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42d13594-12eb-4f2c-8ce7-6c92cfec635d-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:12.961768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:12.961652    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f5qg\" (UniqueName: \"kubernetes.io/projected/42d13594-12eb-4f2c-8ce7-6c92cfec635d-kube-api-access-6f5qg\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:13.378345 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.378312    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"
Oct 29 23:34:13.378805 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.378340    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx" event={"ID":"be7a7c5a-d8b8-4f9d-86f2-435dc7146711","Type":"ContainerDied","Data":"9fed2c8742727fdbc4127a8ba0dbdfeac637b9841e3a0b1ab5c3bfa5625ac7c7"}
Oct 29 23:34:13.378805 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.378407    6040 scope.go:122] "RemoveContainer" containerID="3bc838ab49f5e603fb7f88a45c37be2291e5e047ca91651a5a81a7336219dc74"
Oct 29 23:34:13.380098 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.380071    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"
Oct 29 23:34:13.380189 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.380074    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn" event={"ID":"42d13594-12eb-4f2c-8ce7-6c92cfec635d","Type":"ContainerDied","Data":"5510e21f2d1227adc856211f837f78be809b3e1e59f454ba95937e0685ffe4ad"}
Oct 29 23:34:13.381622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.381605    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"
Oct 29 23:34:13.381790 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.381604    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z" event={"ID":"9fd7b03f-a363-415c-9397-5d202774916d","Type":"ContainerDied","Data":"c519b56bb3632f202a6dd5c1133f8c4523b1a321a4667264e7c521a5ce2a9181"}
Oct 29 23:34:13.391819 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.391797    6040 scope.go:122] "RemoveContainer" containerID="e8e95ceaff603a3321ed7649b720b89f79c599b719da41e6af30812bcf23367c"
Oct 29 23:34:13.404411 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.404390    6040 scope.go:122] "RemoveContainer" containerID="4daa2be0fbcf31fb559427af6f300a038f84b66e5c8d3d5f4a19091c6ae4dafd"
Oct 29 23:34:13.431497 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.431463    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"]
Oct 29 23:34:13.443334 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.443300    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-4ks5z"]
Oct 29 23:34:13.487272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.487236    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"]
Oct 29 23:34:13.491548 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.491516    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-tvljx"]
Oct 29 23:34:13.525935 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.525884    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"]
Oct 29 23:34:13.531343 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:13.531308    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-lr5cn"]
Oct 29 23:34:14.426522 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:14.426493    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" path="/var/lib/kubelet/pods/42d13594-12eb-4f2c-8ce7-6c92cfec635d/volumes"
Oct 29 23:34:14.427021 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:14.427007    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fd7b03f-a363-415c-9397-5d202774916d" path="/var/lib/kubelet/pods/9fd7b03f-a363-415c-9397-5d202774916d/volumes"
Oct 29 23:34:14.427408 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:14.427395    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" path="/var/lib/kubelet/pods/be7a7c5a-d8b8-4f9d-86f2-435dc7146711/volumes"
Oct 29 23:34:16.078717 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.078675    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"]
Oct 29 23:34:16.079406 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.079385    6040 cpu_manager.go:414] "containerMap: removing container" podUID="9fd7b03f-a363-415c-9397-5d202774916d" containerName="istio-proxy"
Oct 29 23:34:16.079511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.079408    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fd7b03f-a363-415c-9397-5d202774916d" containerName="istio-proxy"
Oct 29 23:34:16.079511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.079448    6040 cpu_manager.go:414] "containerMap: removing container" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerName="istio-proxy"
Oct 29 23:34:16.079511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.079456    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerName="istio-proxy"
Oct 29 23:34:16.079511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.079467    6040 cpu_manager.go:414] "containerMap: removing container" podUID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" containerName="istio-proxy"
Oct 29 23:34:16.079511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.079475    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" containerName="istio-proxy"
Oct 29 23:34:16.079786 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.079621    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fd7b03f-a363-415c-9397-5d202774916d" containerName="istio-proxy"
Oct 29 23:34:16.079786 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.079641    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="be7a7c5a-d8b8-4f9d-86f2-435dc7146711" containerName="istio-proxy"
Oct 29 23:34:16.079786 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.079651    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="42d13594-12eb-4f2c-8ce7-6c92cfec635d" containerName="istio-proxy"
Oct 29 23:34:16.084902 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.084879    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:16.102109 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.102080    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"]
Oct 29 23:34:16.109804 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.109728    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:34:16.109971 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.109872    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:34:16.201824 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.201789    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ab06168f-2qw2h\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:16.202018 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.201876    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ab06168f-2qw2h\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:16.202018 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.202004    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf5rf\" (UniqueName: \"kubernetes.io/projected/51061993-4d39-4fab-9f1d-ba0222fa7e6c-kube-api-access-bf5rf\") pod \"maas-api-revoke-sub-ab06168f-2qw2h\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:16.303805 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.303759    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ab06168f-2qw2h\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:16.303981 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.303884    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ab06168f-2qw2h\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:16.304063 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.304028    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf5rf\" (UniqueName: \"kubernetes.io/projected/51061993-4d39-4fab-9f1d-ba0222fa7e6c-kube-api-access-bf5rf\") pod \"maas-api-revoke-sub-ab06168f-2qw2h\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:16.304635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.304606    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ab06168f-2qw2h\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:16.306569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.306542    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ab06168f-2qw2h\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:16.328024 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.328001    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf5rf\" (UniqueName: \"kubernetes.io/projected/51061993-4d39-4fab-9f1d-ba0222fa7e6c-kube-api-access-bf5rf\") pod \"maas-api-revoke-sub-ab06168f-2qw2h\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:16.398972 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.398935    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:16.712813 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:16.712783    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"]
Oct 29 23:34:16.714062 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:34:16.714024    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51061993_4d39_4fab_9f1d_ba0222fa7e6c.slice/crio-066e4784d94787e70586f0591aae06e1af9ec293c77f62d1750cdb87e701dcd0 WatchSource:0}: Error finding container 066e4784d94787e70586f0591aae06e1af9ec293c77f62d1750cdb87e701dcd0: Status 404 returned error can't find the container with id 066e4784d94787e70586f0591aae06e1af9ec293c77f62d1750cdb87e701dcd0
Oct 29 23:34:17.430159 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:17.430120    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h" event={"ID":"51061993-4d39-4fab-9f1d-ba0222fa7e6c","Type":"ContainerStarted","Data":"fa256f9ce222adfb9d6aabbbf0ad2901e99e4cd9f2467e59f48af66e493f8754"}
Oct 29 23:34:17.430559 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:17.430187    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h" event={"ID":"51061993-4d39-4fab-9f1d-ba0222fa7e6c","Type":"ContainerStarted","Data":"066e4784d94787e70586f0591aae06e1af9ec293c77f62d1750cdb87e701dcd0"}
Oct 29 23:34:17.474112 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:17.474058    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h" podStartSLOduration=1.474041089 podStartE2EDuration="1.474041089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:34:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:34:17.472267141 +0000 UTC m=+1237.167376020" watchObservedRunningTime="2026-10-29 23:34:17.474041089 +0000 UTC m=+1237.169149970"
Oct 29 23:34:18.021696 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:18.021660    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"]
Oct 29 23:34:18.050985 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:18.050951    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"]
Oct 29 23:34:18.438055 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:18.438023    6040 generic.go:356] "Generic (PLEG): container finished" podID="51061993-4d39-4fab-9f1d-ba0222fa7e6c" containerID="fa256f9ce222adfb9d6aabbbf0ad2901e99e4cd9f2467e59f48af66e493f8754" exitCode=0
Oct 29 23:34:18.438465 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:18.438102    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h" event={"ID":"51061993-4d39-4fab-9f1d-ba0222fa7e6c","Type":"ContainerDied","Data":"fa256f9ce222adfb9d6aabbbf0ad2901e99e4cd9f2467e59f48af66e493f8754"}
Oct 29 23:34:18.518329 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:18.518289    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:34:18.529879 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:18.529851    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:34:19.591381 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:19.591354    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:19.746228 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:19.746143    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-openshift-ca\") pod \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") "
Oct 29 23:34:19.746228 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:19.746189    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-xks-ca\") pod \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") "
Oct 29 23:34:19.746446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:19.746250    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51061993-4d39-4fab-9f1d-ba0222fa7e6c-kube-api-access-bf5rf\" (UniqueName: \"kubernetes.io/projected/51061993-4d39-4fab-9f1d-ba0222fa7e6c-kube-api-access-bf5rf\") pod \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\" (UID: \"51061993-4d39-4fab-9f1d-ba0222fa7e6c\") "
Oct 29 23:34:19.746608 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:19.746571    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-openshift-ca" pod "51061993-4d39-4fab-9f1d-ba0222fa7e6c" (UID: "51061993-4d39-4fab-9f1d-ba0222fa7e6c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:34:19.746835 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:19.746815    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:19.748415 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:19.748386    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-xks-ca" pod "51061993-4d39-4fab-9f1d-ba0222fa7e6c" (UID: "51061993-4d39-4fab-9f1d-ba0222fa7e6c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:34:19.748496 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:19.748481    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51061993-4d39-4fab-9f1d-ba0222fa7e6c-kube-api-access-bf5rf" pod "51061993-4d39-4fab-9f1d-ba0222fa7e6c" (UID: "51061993-4d39-4fab-9f1d-ba0222fa7e6c"). InnerVolumeSpecName "kube-api-access-bf5rf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:34:19.849212 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:19.849155    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51061993-4d39-4fab-9f1d-ba0222fa7e6c-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:19.849212 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:19.849195    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf5rf\" (UniqueName: \"kubernetes.io/projected/51061993-4d39-4fab-9f1d-ba0222fa7e6c-kube-api-access-bf5rf\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:20.457710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:20.456821    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h" event={"ID":"51061993-4d39-4fab-9f1d-ba0222fa7e6c","Type":"ContainerDied","Data":"066e4784d94787e70586f0591aae06e1af9ec293c77f62d1750cdb87e701dcd0"}
Oct 29 23:34:20.457710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:20.456887    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="066e4784d94787e70586f0591aae06e1af9ec293c77f62d1750cdb87e701dcd0"
Oct 29 23:34:20.470760 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:20.465601    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"
Oct 29 23:34:24.184070 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.184026    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"]
Oct 29 23:34:24.369704 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.369668    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"]
Oct 29 23:34:24.370470 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.370452    6040 cpu_manager.go:414] "containerMap: removing container" podUID="51061993-4d39-4fab-9f1d-ba0222fa7e6c" containerName="revoke-keys"
Oct 29 23:34:24.370557 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.370473    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51061993-4d39-4fab-9f1d-ba0222fa7e6c" containerName="revoke-keys"
Oct 29 23:34:24.370619 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.370591    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="51061993-4d39-4fab-9f1d-ba0222fa7e6c" containerName="revoke-keys"
Oct 29 23:34:24.374652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.374627    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.396560 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.396523    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"]
Oct 29 23:34:24.402801 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.402188    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.402801 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.402243    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.402801 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.402302    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.402801 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.402360    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.402801 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.402401    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.402801 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.402452    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.402801 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.402480    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.402801 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.402535    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.402801 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.402585    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fmdl\" (UniqueName: \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-kube-api-access-4fmdl\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.409350 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.409318    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"]
Oct 29 23:34:24.413829 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.413804    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.477115 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.477086    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"]
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.504521    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.504592    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.504619    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcfb74ac-619a-419e-8284-25b3f1a061b9-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.504877    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.505405    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.505444    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.505490    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.505572    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.505602    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.505664    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.505703    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d595d\" (UniqueName: \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-kube-api-access-d595d\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.505726    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.504567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.505816    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.506800 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.506189    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.506800 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.506308    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.506924 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.506841    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.506985 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.506958    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.507229 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.507202    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.507337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.507257    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.507337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.507317    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.507454 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.507419    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.507508 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.507477    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fmdl\" (UniqueName: \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-kube-api-access-4fmdl\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.507561 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.507530    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.509500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.509473    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.509783 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.509729    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.532710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.532673    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fmdl\" (UniqueName: \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-kube-api-access-4fmdl\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.533529 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.533499    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.608638 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.608601    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.608853 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.608645    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.608853 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.608678    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.608853 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.608695    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcfb74ac-619a-419e-8284-25b3f1a061b9-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.608853 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.608715    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.608853 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.608730    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.608853 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.608830    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.609191 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.608917    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d595d\" (UniqueName: \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-kube-api-access-d595d\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.609191 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.609002    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.609191 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.609099    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.609191 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.609182    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.609349 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.609218    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.609406 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.609387    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.609482 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.609466    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcfb74ac-619a-419e-8284-25b3f1a061b9-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.611267 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.611246    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.611435 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.611417    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.635049 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.635012    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.635211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.635061    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d595d\" (UniqueName: \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-kube-api-access-d595d\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:24.689297 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.689265    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:24.780081 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:24.779798    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:25.110655 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.110630    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"]
Oct 29 23:34:25.111792 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:34:25.111765    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c0a8957_f185_4646_bf8e_7c03d2fa0da2.slice/crio-21db2e3452f67da4397c06aa95626cff9954795df81f8a114838d9cfccad44fa WatchSource:0}: Error finding container 21db2e3452f67da4397c06aa95626cff9954795df81f8a114838d9cfccad44fa: Status 404 returned error can't find the container with id 21db2e3452f67da4397c06aa95626cff9954795df81f8a114838d9cfccad44fa
Oct 29 23:34:25.114057 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.114024    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:34:25.114129 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.114088    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:34:25.114129 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.114117    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:34:25.124140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.124118    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"]
Oct 29 23:34:25.125924 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:34:25.125904    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfb74ac_619a_419e_8284_25b3f1a061b9.slice/crio-cec9318beac1776644af158ecbd78b8db506122e2deef0b6b75e1b34f9cba75e WatchSource:0}: Error finding container cec9318beac1776644af158ecbd78b8db506122e2deef0b6b75e1b34f9cba75e: Status 404 returned error can't find the container with id cec9318beac1776644af158ecbd78b8db506122e2deef0b6b75e1b34f9cba75e
Oct 29 23:34:25.129014 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.128975    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:34:25.129088 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.129058    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:34:25.129124 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.129098    6040 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 29 23:34:25.494932 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.494885    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" event={"ID":"dcfb74ac-619a-419e-8284-25b3f1a061b9","Type":"ContainerStarted","Data":"a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496"}
Oct 29 23:34:25.495392 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.494960    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" event={"ID":"dcfb74ac-619a-419e-8284-25b3f1a061b9","Type":"ContainerStarted","Data":"cec9318beac1776644af158ecbd78b8db506122e2deef0b6b75e1b34f9cba75e"}
Oct 29 23:34:25.496712 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.496684    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" event={"ID":"2c0a8957-f185-4646-bf8e-7c03d2fa0da2","Type":"ContainerStarted","Data":"2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1"}
Oct 29 23:34:25.496849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.496732    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" event={"ID":"2c0a8957-f185-4646-bf8e-7c03d2fa0da2","Type":"ContainerStarted","Data":"21db2e3452f67da4397c06aa95626cff9954795df81f8a114838d9cfccad44fa"}
Oct 29 23:34:25.542037 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.541958    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" podStartSLOduration=1.541939905 podStartE2EDuration="1.541939905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:34:25.540621845 +0000 UTC m=+1245.235730739" watchObservedRunningTime="2026-10-29 23:34:25.541939905 +0000 UTC m=+1245.237048788"
Oct 29 23:34:25.592380 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.592307    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" podStartSLOduration=1.5922858469999999 podStartE2EDuration="1.592285847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:34:25.591653132 +0000 UTC m=+1245.286762013" watchObservedRunningTime="2026-10-29 23:34:25.592285847 +0000 UTC m=+1245.287394727"
Oct 29 23:34:25.690118 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.690081    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:25.780998 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:25.780916    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:26.690700 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:26.690667    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 29 23:34:26.691121 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:26.690719    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" podUID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 29 23:34:26.782322 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:26.782280    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 29 23:34:26.782468 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:26.782332    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" podUID="dcfb74ac-619a-419e-8284-25b3f1a061b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 29 23:34:27.689625 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:27.689592    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 29 23:34:27.689967 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:27.689936    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" podUID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 29 23:34:27.781077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:27.781036    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 29 23:34:27.781556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:27.781099    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" podUID="dcfb74ac-619a-419e-8284-25b3f1a061b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 29 23:34:28.690273 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:28.690239    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 29 23:34:28.690443 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:28.690291    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" podUID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 29 23:34:28.780653 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:28.780615    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 29 23:34:28.780833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:28.780664    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" podUID="dcfb74ac-619a-419e-8284-25b3f1a061b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 29 23:34:29.184792 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:29.184732    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg" podUID="a539d29d-cb81-47d7-863d-e59027a12b58" containerName="payload-processing" containerID="cri-o://e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d" gracePeriod=25
Oct 29 23:34:29.690195 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:29.690163    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 29 23:34:29.690401 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:29.690230    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" podUID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 29 23:34:29.780596 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:34:29.780557    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:34:29.780886 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:29.780866    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 29 23:34:29.780936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:29.780911    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" podUID="dcfb74ac-619a-419e-8284-25b3f1a061b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 29 23:34:30.690421 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:30.690391    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 29 23:34:30.690868 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:30.690440    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" podUID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 29 23:34:30.781504 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:30.781467    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 29 23:34:30.781666 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:30.781522    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" podUID="dcfb74ac-619a-419e-8284-25b3f1a061b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 29 23:34:31.689917 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:31.689878    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 29 23:34:31.690129 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:31.689938    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" podUID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 29 23:34:31.781528 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:31.781483    6040 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 29 23:34:31.782034 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:31.781548    6040 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" podUID="dcfb74ac-619a-419e-8284-25b3f1a061b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 29 23:34:32.675691 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.675646    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"]
Oct 29 23:34:32.680706 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.680671    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:32.698439 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.698405    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"]
Oct 29 23:34:32.716029 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.715996    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:34:32.716207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.716115    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:34:32.731695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.731666    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:32.732990 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.732963    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:32.735318 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.735294    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:34:32.785333 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.785306    6040 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:32.785694 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.785579    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:32.786147 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.786129    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:34:32.789782 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.789760    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e30686a-4zpwl\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:32.789858 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.789794    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e30686a-4zpwl\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:32.789858 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.789814    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzrsf\" (UniqueName: \"kubernetes.io/projected/ccb7ea93-728f-46a0-a1b0-1358aab430ee-kube-api-access-dzrsf\") pod \"maas-api-revoke-sub-2e30686a-4zpwl\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:32.891246 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.891206    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e30686a-4zpwl\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:32.891441 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.891268    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e30686a-4zpwl\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:32.891441 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.891297    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzrsf\" (UniqueName: \"kubernetes.io/projected/ccb7ea93-728f-46a0-a1b0-1358aab430ee-kube-api-access-dzrsf\") pod \"maas-api-revoke-sub-2e30686a-4zpwl\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:32.892027 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.891997    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e30686a-4zpwl\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:32.893958 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.893930    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e30686a-4zpwl\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:32.924198 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:32.924160    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzrsf\" (UniqueName: \"kubernetes.io/projected/ccb7ea93-728f-46a0-a1b0-1358aab430ee-kube-api-access-dzrsf\") pod \"maas-api-revoke-sub-2e30686a-4zpwl\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:33.009784 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.009690    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:33.043162 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.043120    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" podUID="07800096-1dc8-41ca-9a09-756653bbf742" containerName="main" containerID="cri-o://15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e" gracePeriod=45
Oct 29 23:34:33.074546 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.074506    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main" containerID="cri-o://c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1" gracePeriod=45
Oct 29 23:34:33.316156 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.316127    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"]
Oct 29 23:34:33.319129 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:34:33.319097    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb7ea93_728f_46a0_a1b0_1358aab430ee.slice/crio-aa94eba99b45f54d196a3d84d803b199264e354028895448678eb335937a0d4c WatchSource:0}: Error finding container aa94eba99b45f54d196a3d84d803b199264e354028895448678eb335937a0d4c: Status 404 returned error can't find the container with id aa94eba99b45f54d196a3d84d803b199264e354028895448678eb335937a0d4c
Oct 29 23:34:33.343132 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.343108    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:34:33.358912 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.358877    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:34:33.499822 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.499786    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-model-cache\") pod \"6f002fa2-9040-468f-9042-6d2c8abec047\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") "
Oct 29 23:34:33.499967 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.499850    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07800096-1dc8-41ca-9a09-756653bbf742-kube-api-access-zx6qv\" (UniqueName: \"kubernetes.io/projected/07800096-1dc8-41ca-9a09-756653bbf742-kube-api-access-zx6qv\") pod \"07800096-1dc8-41ca-9a09-756653bbf742\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") "
Oct 29 23:34:33.499967 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.499910    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-tmp-dir\") pod \"6f002fa2-9040-468f-9042-6d2c8abec047\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") "
Oct 29 23:34:33.499967 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.499949    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-dshm\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-dshm\") pod \"07800096-1dc8-41ca-9a09-756653bbf742\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") "
Oct 29 23:34:33.500120 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.499981    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f002fa2-9040-468f-9042-6d2c8abec047-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f002fa2-9040-468f-9042-6d2c8abec047-tls-certs\") pod \"6f002fa2-9040-468f-9042-6d2c8abec047\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") "
Oct 29 23:34:33.500120 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500021    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-tmp-dir\") pod \"07800096-1dc8-41ca-9a09-756653bbf742\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") "
Oct 29 23:34:33.500120 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500084    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07800096-1dc8-41ca-9a09-756653bbf742-tls-certs\" (UniqueName: \"kubernetes.io/secret/07800096-1dc8-41ca-9a09-756653bbf742-tls-certs\") pod \"07800096-1dc8-41ca-9a09-756653bbf742\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") "
Oct 29 23:34:33.500120 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500093    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-model-cache" pod "6f002fa2-9040-468f-9042-6d2c8abec047" (UID: "6f002fa2-9040-468f-9042-6d2c8abec047"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:33.500329 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500156    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-dshm\") pod \"6f002fa2-9040-468f-9042-6d2c8abec047\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") "
Oct 29 23:34:33.500329 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500192    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-home\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-home\") pod \"6f002fa2-9040-468f-9042-6d2c8abec047\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") "
Oct 29 23:34:33.500329 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500312    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-model-cache\") pod \"07800096-1dc8-41ca-9a09-756653bbf742\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") "
Oct 29 23:34:33.500592 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500287    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-tmp-dir" pod "6f002fa2-9040-468f-9042-6d2c8abec047" (UID: "6f002fa2-9040-468f-9042-6d2c8abec047"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:33.500592 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500346    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-home\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-home\") pod \"07800096-1dc8-41ca-9a09-756653bbf742\" (UID: \"07800096-1dc8-41ca-9a09-756653bbf742\") "
Oct 29 23:34:33.500592 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500429    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f002fa2-9040-468f-9042-6d2c8abec047-kube-api-access-nmk78\" (UniqueName: \"kubernetes.io/projected/6f002fa2-9040-468f-9042-6d2c8abec047-kube-api-access-nmk78\") pod \"6f002fa2-9040-468f-9042-6d2c8abec047\" (UID: \"6f002fa2-9040-468f-9042-6d2c8abec047\") "
Oct 29 23:34:33.500592 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500566    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-home" pod "07800096-1dc8-41ca-9a09-756653bbf742" (UID: "07800096-1dc8-41ca-9a09-756653bbf742"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:33.500850 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500599    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-home" pod "6f002fa2-9040-468f-9042-6d2c8abec047" (UID: "6f002fa2-9040-468f-9042-6d2c8abec047"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:33.500850 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500608    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-tmp-dir" pod "07800096-1dc8-41ca-9a09-756653bbf742" (UID: "07800096-1dc8-41ca-9a09-756653bbf742"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:33.500850 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.500777    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-model-cache" pod "07800096-1dc8-41ca-9a09-756653bbf742" (UID: "07800096-1dc8-41ca-9a09-756653bbf742"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:33.502218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.501120    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.502218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.501145    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.502218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.501158    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.502218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.501171    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.502218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.501184    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.502218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.501197    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.502802 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.502775    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f002fa2-9040-468f-9042-6d2c8abec047-tls-certs" pod "6f002fa2-9040-468f-9042-6d2c8abec047" (UID: "6f002fa2-9040-468f-9042-6d2c8abec047"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:34:33.503149 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.503097    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07800096-1dc8-41ca-9a09-756653bbf742-kube-api-access-zx6qv" pod "07800096-1dc8-41ca-9a09-756653bbf742" (UID: "07800096-1dc8-41ca-9a09-756653bbf742"). InnerVolumeSpecName "kube-api-access-zx6qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:34:33.503251 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.503233    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-dshm" pod "6f002fa2-9040-468f-9042-6d2c8abec047" (UID: "6f002fa2-9040-468f-9042-6d2c8abec047"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:33.503378 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.503353    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-dshm" pod "07800096-1dc8-41ca-9a09-756653bbf742" (UID: "07800096-1dc8-41ca-9a09-756653bbf742"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:34:33.503430 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.503371    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07800096-1dc8-41ca-9a09-756653bbf742-tls-certs" pod "07800096-1dc8-41ca-9a09-756653bbf742" (UID: "07800096-1dc8-41ca-9a09-756653bbf742"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:34:33.504043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.504017    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f002fa2-9040-468f-9042-6d2c8abec047-kube-api-access-nmk78" pod "6f002fa2-9040-468f-9042-6d2c8abec047" (UID: "6f002fa2-9040-468f-9042-6d2c8abec047"). InnerVolumeSpecName "kube-api-access-nmk78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:34:33.602510 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.602473    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07800096-1dc8-41ca-9a09-756653bbf742-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.602510 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.602502    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f002fa2-9040-468f-9042-6d2c8abec047-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.602510 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.602517    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07800096-1dc8-41ca-9a09-756653bbf742-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.602730 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.602527    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f002fa2-9040-468f-9042-6d2c8abec047-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.602730 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.602536    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmk78\" (UniqueName: \"kubernetes.io/projected/6f002fa2-9040-468f-9042-6d2c8abec047-kube-api-access-nmk78\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.602730 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.602544    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zx6qv\" (UniqueName: \"kubernetes.io/projected/07800096-1dc8-41ca-9a09-756653bbf742-kube-api-access-zx6qv\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:33.620173 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.620137    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl" event={"ID":"ccb7ea93-728f-46a0-a1b0-1358aab430ee","Type":"ContainerStarted","Data":"aa94eba99b45f54d196a3d84d803b199264e354028895448678eb335937a0d4c"}
Oct 29 23:34:33.621799 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.621732    6040 generic.go:356] "Generic (PLEG): container finished" podID="6f002fa2-9040-468f-9042-6d2c8abec047" containerID="c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1" exitCode=0
Oct 29 23:34:33.621909 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.621824    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"
Oct 29 23:34:33.621909 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.621829    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" event={"ID":"6f002fa2-9040-468f-9042-6d2c8abec047","Type":"ContainerDied","Data":"c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1"}
Oct 29 23:34:33.621909 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.621874    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk" event={"ID":"6f002fa2-9040-468f-9042-6d2c8abec047","Type":"ContainerDied","Data":"e84d68c8180623e76bd0ba7484f4bb5b86e7bc9c2f000021481849c885b8c050"}
Oct 29 23:34:33.621909 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.621894    6040 scope.go:122] "RemoveContainer" containerID="c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1"
Oct 29 23:34:33.623609 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.623586    6040 generic.go:356] "Generic (PLEG): container finished" podID="07800096-1dc8-41ca-9a09-756653bbf742" containerID="15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e" exitCode=0
Oct 29 23:34:33.623693 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.623638    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"
Oct 29 23:34:33.623798 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.623638    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" event={"ID":"07800096-1dc8-41ca-9a09-756653bbf742","Type":"ContainerDied","Data":"15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e"}
Oct 29 23:34:33.623798 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.623778    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c" event={"ID":"07800096-1dc8-41ca-9a09-756653bbf742","Type":"ContainerDied","Data":"9a195185dcccd263ed1cff2bcd1fe80a51fb6ee9f591025f37c7ee002073abc5"}
Oct 29 23:34:33.632651 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.632633    6040 scope.go:122] "RemoveContainer" containerID="c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1"
Oct 29 23:34:33.632960 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:34:33.632936    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1\": container with ID starting with c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1 not found: ID does not exist" containerID="c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1"
Oct 29 23:34:33.633021 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.632973    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1"} err="failed to get container status \"c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1\": rpc error: code = NotFound desc = could not find container \"c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1\": container with ID starting with c506e70aa7ae38f46f1d2eb1c67c3b5959ac3b0c8fe95b083a127e92a377a4f1 not found: ID does not exist"
Oct 29 23:34:33.633021 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.632999    6040 scope.go:122] "RemoveContainer" containerID="15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e"
Oct 29 23:34:33.641195 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.641162    6040 scope.go:122] "RemoveContainer" containerID="15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e"
Oct 29 23:34:33.641410 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:34:33.641394    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e\": container with ID starting with 15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e not found: ID does not exist" containerID="15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e"
Oct 29 23:34:33.641446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.641416    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e"} err="failed to get container status \"15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e\": rpc error: code = NotFound desc = could not find container \"15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e\": container with ID starting with 15aa9b0fecaa9b64adc641e75ae45b3d468e72bdf86c68cbaab4cbb0e2c4552e not found: ID does not exist"
Oct 29 23:34:33.675819 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.675795    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"]
Oct 29 23:34:33.678872 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.678851    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-2-w1-2a53c9-kserve-5d978555f8-w9n2c"]
Oct 29 23:34:33.711865 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.711835    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"]
Oct 29 23:34:33.716919 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:33.716895    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/simulator-w1-2a53c9-kserve-749cd577bc-snldk"]
Oct 29 23:34:34.425776 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:34.425673    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07800096-1dc8-41ca-9a09-756653bbf742" path="/var/lib/kubelet/pods/07800096-1dc8-41ca-9a09-756653bbf742/volumes"
Oct 29 23:34:34.426342 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:34.426320    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" path="/var/lib/kubelet/pods/6f002fa2-9040-468f-9042-6d2c8abec047/volumes"
Oct 29 23:34:34.643094 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:34.643060    6040 generic.go:356] "Generic (PLEG): container finished" podID="ccb7ea93-728f-46a0-a1b0-1358aab430ee" containerID="9149bb6d1b243e805f95dc72c19abcd7ebcb407c7cac0c7816deecba3be590e3" exitCode=0
Oct 29 23:34:34.643275 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:34.643117    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl" event={"ID":"ccb7ea93-728f-46a0-a1b0-1358aab430ee","Type":"ContainerDied","Data":"9149bb6d1b243e805f95dc72c19abcd7ebcb407c7cac0c7816deecba3be590e3"}
Oct 29 23:34:35.743160 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:35.743126    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:34:35.805754 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:35.805721    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:35.931347 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:35.931152    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ccb7ea93-728f-46a0-a1b0-1358aab430ee-kube-api-access-dzrsf\" (UniqueName: \"kubernetes.io/projected/ccb7ea93-728f-46a0-a1b0-1358aab430ee-kube-api-access-dzrsf\") pod \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") "
Oct 29 23:34:35.931347 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:35.931205    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-xks-ca\") pod \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") "
Oct 29 23:34:35.931347 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:35.931291    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-openshift-ca\") pod \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\" (UID: \"ccb7ea93-728f-46a0-a1b0-1358aab430ee\") "
Oct 29 23:34:35.931946 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:35.931801    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-openshift-ca" pod "ccb7ea93-728f-46a0-a1b0-1358aab430ee" (UID: "ccb7ea93-728f-46a0-a1b0-1358aab430ee"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:34:35.933687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:35.933658    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccb7ea93-728f-46a0-a1b0-1358aab430ee-kube-api-access-dzrsf" pod "ccb7ea93-728f-46a0-a1b0-1358aab430ee" (UID: "ccb7ea93-728f-46a0-a1b0-1358aab430ee"). InnerVolumeSpecName "kube-api-access-dzrsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:34:35.934082 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:35.934050    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-xks-ca" pod "ccb7ea93-728f-46a0-a1b0-1358aab430ee" (UID: "ccb7ea93-728f-46a0-a1b0-1358aab430ee"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:34:36.032572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:36.032542    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:36.032572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:36.032569    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzrsf\" (UniqueName: \"kubernetes.io/projected/ccb7ea93-728f-46a0-a1b0-1358aab430ee-kube-api-access-dzrsf\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:36.032572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:36.032580    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ccb7ea93-728f-46a0-a1b0-1358aab430ee-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:36.658723 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:36.658693    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"
Oct 29 23:34:36.658723 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:36.658703    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl" event={"ID":"ccb7ea93-728f-46a0-a1b0-1358aab430ee","Type":"ContainerDied","Data":"aa94eba99b45f54d196a3d84d803b199264e354028895448678eb335937a0d4c"}
Oct 29 23:34:36.659055 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:36.658784    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa94eba99b45f54d196a3d84d803b199264e354028895448678eb335937a0d4c"
Oct 29 23:34:38.082426 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:38.082387    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg" podUID="a539d29d-cb81-47d7-863d-e59027a12b58" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 29 23:34:48.082284 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:48.082243    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg" podUID="a539d29d-cb81-47d7-863d-e59027a12b58" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 29 23:34:49.037095 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.037061    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"]
Oct 29 23:34:49.049136 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.049105    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"]
Oct 29 23:34:49.463255 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.463226    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:34:49.522784 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.522752    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a539d29d-cb81-47d7-863d-e59027a12b58-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a539d29d-cb81-47d7-863d-e59027a12b58-plugins-config-volume\") pod \"a539d29d-cb81-47d7-863d-e59027a12b58\" (UID: \"a539d29d-cb81-47d7-863d-e59027a12b58\") "
Oct 29 23:34:49.522972 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.522898    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a539d29d-cb81-47d7-863d-e59027a12b58-kube-api-access-2h6x5\" (UniqueName: \"kubernetes.io/projected/a539d29d-cb81-47d7-863d-e59027a12b58-kube-api-access-2h6x5\") pod \"a539d29d-cb81-47d7-863d-e59027a12b58\" (UID: \"a539d29d-cb81-47d7-863d-e59027a12b58\") "
Oct 29 23:34:49.523162 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.523133    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a539d29d-cb81-47d7-863d-e59027a12b58-plugins-config-volume" pod "a539d29d-cb81-47d7-863d-e59027a12b58" (UID: "a539d29d-cb81-47d7-863d-e59027a12b58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:34:49.523413 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.523388    6040 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a539d29d-cb81-47d7-863d-e59027a12b58-plugins-config-volume\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:49.525071 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.525042    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a539d29d-cb81-47d7-863d-e59027a12b58-kube-api-access-2h6x5" pod "a539d29d-cb81-47d7-863d-e59027a12b58" (UID: "a539d29d-cb81-47d7-863d-e59027a12b58"). InnerVolumeSpecName "kube-api-access-2h6x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:34:49.625575 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.625493    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2h6x5\" (UniqueName: \"kubernetes.io/projected/a539d29d-cb81-47d7-863d-e59027a12b58-kube-api-access-2h6x5\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:34:49.757967 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.757932    6040 generic.go:356] "Generic (PLEG): container finished" podID="a539d29d-cb81-47d7-863d-e59027a12b58" containerID="e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d" exitCode=0
Oct 29 23:34:49.758166 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.757987    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"
Oct 29 23:34:49.758166 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.758011    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg" event={"ID":"a539d29d-cb81-47d7-863d-e59027a12b58","Type":"ContainerDied","Data":"e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d"}
Oct 29 23:34:49.758166 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.758101    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg" event={"ID":"a539d29d-cb81-47d7-863d-e59027a12b58","Type":"ContainerDied","Data":"458962e60a173f3a23dc1cd2379ac58a852d6251b1084028075019fff2a1cca3"}
Oct 29 23:34:49.758166 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.758127    6040 scope.go:122] "RemoveContainer" containerID="e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d"
Oct 29 23:34:49.768627 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.768609    6040 scope.go:122] "RemoveContainer" containerID="e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d"
Oct 29 23:34:49.768895 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:34:49.768876    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d\": container with ID starting with e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d not found: ID does not exist" containerID="e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d"
Oct 29 23:34:49.768947 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.768902    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d"} err="failed to get container status \"e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d\": rpc error: code = NotFound desc = could not find container \"e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d\": container with ID starting with e9fd7f4f019450289eda6c37273cc0d66167123be37f68aa52e72387c260140d not found: ID does not exist"
Oct 29 23:34:49.810339 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.810308    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"]
Oct 29 23:34:49.822268 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:49.822237    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-2a53c9-b779d778-s8ghg"]
Oct 29 23:34:50.425368 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:50.425334    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a539d29d-cb81-47d7-863d-e59027a12b58" path="/var/lib/kubelet/pods/a539d29d-cb81-47d7-863d-e59027a12b58/volumes"
Oct 29 23:34:55.739406 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:55.739367    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"]
Oct 29 23:34:55.739932 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:55.739631    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" podUID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" containerName="istio-proxy" containerID="cri-o://9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29" gracePeriod=30
Oct 29 23:34:55.768345 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:55.768318    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"]
Oct 29 23:34:55.768548 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:55.768528    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" podUID="dcfb74ac-619a-419e-8284-25b3f1a061b9" containerName="istio-proxy" containerID="cri-o://a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496" gracePeriod=30
Oct 29 23:34:55.777584 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:55.777558    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"]
Oct 29 23:34:55.777795 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:55.777769    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" podUID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" containerName="istio-proxy" containerID="cri-o://2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1" gracePeriod=30
Oct 29 23:34:57.834383 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:57.834346    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"]
Oct 29 23:34:59.699119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699083    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"]
Oct 29 23:34:59.699572 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699558    6040 cpu_manager.go:414] "containerMap: removing container" podUID="ccb7ea93-728f-46a0-a1b0-1358aab430ee" containerName="revoke-keys"
Oct 29 23:34:59.699620 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699573    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ccb7ea93-728f-46a0-a1b0-1358aab430ee" containerName="revoke-keys"
Oct 29 23:34:59.699620 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699605    6040 cpu_manager.go:414] "containerMap: removing container" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main"
Oct 29 23:34:59.699620 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699610    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main"
Oct 29 23:34:59.699716 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699621    6040 cpu_manager.go:414] "containerMap: removing container" podUID="07800096-1dc8-41ca-9a09-756653bbf742" containerName="main"
Oct 29 23:34:59.699716 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699626    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07800096-1dc8-41ca-9a09-756653bbf742" containerName="main"
Oct 29 23:34:59.699716 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699637    6040 cpu_manager.go:414] "containerMap: removing container" podUID="a539d29d-cb81-47d7-863d-e59027a12b58" containerName="payload-processing"
Oct 29 23:34:59.699716 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699642    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a539d29d-cb81-47d7-863d-e59027a12b58" containerName="payload-processing"
Oct 29 23:34:59.699716 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699708    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="a539d29d-cb81-47d7-863d-e59027a12b58" containerName="payload-processing"
Oct 29 23:34:59.699716 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699716    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="ccb7ea93-728f-46a0-a1b0-1358aab430ee" containerName="revoke-keys"
Oct 29 23:34:59.699933 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699727    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f002fa2-9040-468f-9042-6d2c8abec047" containerName="main"
Oct 29 23:34:59.699933 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.699734    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="07800096-1dc8-41ca-9a09-756653bbf742" containerName="main"
Oct 29 23:34:59.703193 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.703172    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:34:59.721616 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.721579    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"]
Oct 29 23:34:59.729565 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.729539    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:34:59.729860 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.729841    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:34:59.780488 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.780405    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcm2k\" (UniqueName: \"kubernetes.io/projected/4a8fbb40-37c6-44ef-8339-5bd0187e687b-kube-api-access-jcm2k\") pod \"maas-api-revoke-sub-9b4dbfcf-zl95g\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:34:59.780660 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:34:59.780564    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:34:59.780864 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.780793    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b4dbfcf-zl95g\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:34:59.780864 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.780837    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b4dbfcf-zl95g\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:34:59.881826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.881789    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcm2k\" (UniqueName: \"kubernetes.io/projected/4a8fbb40-37c6-44ef-8339-5bd0187e687b-kube-api-access-jcm2k\") pod \"maas-api-revoke-sub-9b4dbfcf-zl95g\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:34:59.882024 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.881850    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b4dbfcf-zl95g\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:34:59.882024 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.881894    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b4dbfcf-zl95g\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:34:59.882510 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.882485    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b4dbfcf-zl95g\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:34:59.884206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.884179    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b4dbfcf-zl95g\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:34:59.905379 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:34:59.905338    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcm2k\" (UniqueName: \"kubernetes.io/projected/4a8fbb40-37c6-44ef-8339-5bd0187e687b-kube-api-access-jcm2k\") pod \"maas-api-revoke-sub-9b4dbfcf-zl95g\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:35:00.016473 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:00.016384    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:35:00.319961 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:00.319932    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"]
Oct 29 23:35:00.321051 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:35:00.321020    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8fbb40_37c6_44ef_8339_5bd0187e687b.slice/crio-7519c98d3c884bf2dffcb4b5c265672877a8fa955f01b58f8f12835ea238b226 WatchSource:0}: Error finding container 7519c98d3c884bf2dffcb4b5c265672877a8fa955f01b58f8f12835ea238b226: Status 404 returned error can't find the container with id 7519c98d3c884bf2dffcb4b5c265672877a8fa955f01b58f8f12835ea238b226
Oct 29 23:35:00.839113 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:00.839078    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g" event={"ID":"4a8fbb40-37c6-44ef-8339-5bd0187e687b","Type":"ContainerStarted","Data":"24d890d55c6df4f05240553cba5f7478f883c6c321f156b7b0862e83fb924cca"}
Oct 29 23:35:00.839522 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:00.839133    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g" event={"ID":"4a8fbb40-37c6-44ef-8339-5bd0187e687b","Type":"ContainerStarted","Data":"7519c98d3c884bf2dffcb4b5c265672877a8fa955f01b58f8f12835ea238b226"}
Oct 29 23:35:00.909532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:00.909462    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g" podStartSLOduration=1.909439337 podStartE2EDuration="1.909439337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:35:00.904386143 +0000 UTC m=+1280.599495024" watchObservedRunningTime="2026-10-29 23:35:00.909439337 +0000 UTC m=+1280.604548217"
Oct 29 23:35:01.082540 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.082514    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:35:01.096944 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.096915    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-envoy\") pod \"dcfb74ac-619a-419e-8284-25b3f1a061b9\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") "
Oct 29 23:35:01.099954 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.099926    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-envoy" pod "dcfb74ac-619a-419e-8284-25b3f1a061b9" (UID: "dcfb74ac-619a-419e-8284-25b3f1a061b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.134378 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.134346    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:35:01.136365 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.136336    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:35:01.198651 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.198613    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-credential-socket\") pod \"dcfb74ac-619a-419e-8284-25b3f1a061b9\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") "
Oct 29 23:35:01.198861 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.198721    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-token\" (UniqueName: \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-token\") pod \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") "
Oct 29 23:35:01.198861 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.198798    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-credential-socket\") pod \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") "
Oct 29 23:35:01.198861 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.198837    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-data\") pod \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") "
Oct 29 23:35:01.199069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.198885    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istiod-ca-cert\") pod \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") "
Oct 29 23:35:01.199069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.198925    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-envoy\") pod \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") "
Oct 29 23:35:01.199069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.198959    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-data\") pod \"dcfb74ac-619a-419e-8284-25b3f1a061b9\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") "
Oct 29 23:35:01.199069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199000    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-certs\") pod \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") "
Oct 29 23:35:01.199069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199034    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-socket\") pod \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") "
Oct 29 23:35:01.199069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199066    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-certs\") pod \"dcfb74ac-619a-419e-8284-25b3f1a061b9\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") "
Oct 29 23:35:01.199410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199097    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-kube-api-access-xbwbn\" (UniqueName: \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-kube-api-access-xbwbn\") pod \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") "
Oct 29 23:35:01.199410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199136    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dcfb74ac-619a-419e-8284-25b3f1a061b9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcfb74ac-619a-419e-8284-25b3f1a061b9-istiod-ca-cert\") pod \"dcfb74ac-619a-419e-8284-25b3f1a061b9\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") "
Oct 29 23:35:01.199410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199169    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-socket\") pod \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") "
Oct 29 23:35:01.199410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199194    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-token\" (UniqueName: \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-token\") pod \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") "
Oct 29 23:35:01.199410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199232    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-podinfo\") pod \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") "
Oct 29 23:35:01.199410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199276    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-podinfo\") pod \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") "
Oct 29 23:35:01.199410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199326    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istiod-ca-cert\") pod \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") "
Oct 29 23:35:01.199410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199378    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-kube-api-access-d595d\" (UniqueName: \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-kube-api-access-d595d\") pod \"dcfb74ac-619a-419e-8284-25b3f1a061b9\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") "
Oct 29 23:35:01.199410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199409    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-data\") pod \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\" (UID: \"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89\") "
Oct 29 23:35:01.199873 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199492    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-socket\") pod \"dcfb74ac-619a-419e-8284-25b3f1a061b9\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") "
Oct 29 23:35:01.199873 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199518    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-certs\") pod \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") "
Oct 29 23:35:01.199873 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199545    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-podinfo\") pod \"dcfb74ac-619a-419e-8284-25b3f1a061b9\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") "
Oct 29 23:35:01.199873 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199566    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-credential-socket\") pod \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") "
Oct 29 23:35:01.199873 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199583    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-token\" (UniqueName: \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-token\") pod \"dcfb74ac-619a-419e-8284-25b3f1a061b9\" (UID: \"dcfb74ac-619a-419e-8284-25b3f1a061b9\") "
Oct 29 23:35:01.199873 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199608    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-kube-api-access-4fmdl\" (UniqueName: \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-kube-api-access-4fmdl\") pod \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") "
Oct 29 23:35:01.199873 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.199632    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-envoy\") pod \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\" (UID: \"2c0a8957-f185-4646-bf8e-7c03d2fa0da2\") "
Oct 29 23:35:01.200219 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.200094    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.200571 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.200542    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-socket" pod "2c0a8957-f185-4646-bf8e-7c03d2fa0da2" (UID: "2c0a8957-f185-4646-bf8e-7c03d2fa0da2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.200691 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.200636    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-data" pod "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" (UID: "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.200823 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.200803    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-data" pod "dcfb74ac-619a-419e-8284-25b3f1a061b9" (UID: "dcfb74ac-619a-419e-8284-25b3f1a061b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.203099 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.203071    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istiod-ca-cert" pod "2c0a8957-f185-4646-bf8e-7c03d2fa0da2" (UID: "2c0a8957-f185-4646-bf8e-7c03d2fa0da2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:01.203261 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.203132    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-data" pod "2c0a8957-f185-4646-bf8e-7c03d2fa0da2" (UID: "2c0a8957-f185-4646-bf8e-7c03d2fa0da2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.203532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.203445    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-credential-socket" pod "dcfb74ac-619a-419e-8284-25b3f1a061b9" (UID: "dcfb74ac-619a-419e-8284-25b3f1a061b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.203881 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.203716    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-certs" pod "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" (UID: "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.204400 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.204008    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-socket" pod "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" (UID: "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.204400 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.204134    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istiod-ca-cert" pod "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" (UID: "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:01.204985 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.204919    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-credential-socket" pod "2c0a8957-f185-4646-bf8e-7c03d2fa0da2" (UID: "2c0a8957-f185-4646-bf8e-7c03d2fa0da2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.205349 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.205166    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-credential-socket" pod "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" (UID: "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.206207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.206123    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-certs" pod "dcfb74ac-619a-419e-8284-25b3f1a061b9" (UID: "dcfb74ac-619a-419e-8284-25b3f1a061b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.206207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.206183    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-certs" pod "2c0a8957-f185-4646-bf8e-7c03d2fa0da2" (UID: "2c0a8957-f185-4646-bf8e-7c03d2fa0da2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.206207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.206180    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcfb74ac-619a-419e-8284-25b3f1a061b9-istiod-ca-cert" pod "dcfb74ac-619a-419e-8284-25b3f1a061b9" (UID: "dcfb74ac-619a-419e-8284-25b3f1a061b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:01.206481 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.206453    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-socket" pod "dcfb74ac-619a-419e-8284-25b3f1a061b9" (UID: "dcfb74ac-619a-419e-8284-25b3f1a061b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.206891 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.206843    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-token" pod "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" (UID: "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:01.206891 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.206880    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-kube-api-access-d595d" pod "dcfb74ac-619a-419e-8284-25b3f1a061b9" (UID: "dcfb74ac-619a-419e-8284-25b3f1a061b9"). InnerVolumeSpecName "kube-api-access-d595d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:01.208140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.208114    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-podinfo" pod "dcfb74ac-619a-419e-8284-25b3f1a061b9" (UID: "dcfb74ac-619a-419e-8284-25b3f1a061b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:01.212075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.212038    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-podinfo" pod "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" (UID: "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:01.212075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.212056    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-token" pod "dcfb74ac-619a-419e-8284-25b3f1a061b9" (UID: "dcfb74ac-619a-419e-8284-25b3f1a061b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:01.212075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.212056    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-podinfo" pod "2c0a8957-f185-4646-bf8e-7c03d2fa0da2" (UID: "2c0a8957-f185-4646-bf8e-7c03d2fa0da2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:01.212293 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.212158    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-token" pod "2c0a8957-f185-4646-bf8e-7c03d2fa0da2" (UID: "2c0a8957-f185-4646-bf8e-7c03d2fa0da2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:01.212293 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.212222    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-envoy" pod "2c0a8957-f185-4646-bf8e-7c03d2fa0da2" (UID: "2c0a8957-f185-4646-bf8e-7c03d2fa0da2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.212293 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.212238    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-envoy" pod "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" (UID: "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:01.212554 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.212527    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-kube-api-access-xbwbn" pod "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" (UID: "a5e71c48-e2a9-4c91-bf79-b5c5baef9c89"). InnerVolumeSpecName "kube-api-access-xbwbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:01.213477 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.213455    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-kube-api-access-4fmdl" pod "2c0a8957-f185-4646-bf8e-7c03d2fa0da2" (UID: "2c0a8957-f185-4646-bf8e-7c03d2fa0da2"). InnerVolumeSpecName "kube-api-access-4fmdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:01.301479 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301436    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301479 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301472    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301479 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301485    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301496    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301509    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301522    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbwbn\" (UniqueName: \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-kube-api-access-xbwbn\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301535    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcfb74ac-619a-419e-8284-25b3f1a061b9-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301546    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301557    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301569    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301579    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301591    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301603    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d595d\" (UniqueName: \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-kube-api-access-d595d\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301617    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301629    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301641    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301653    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301664    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301676    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcfb74ac-619a-419e-8284-25b3f1a061b9-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301688    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fmdl\" (UniqueName: \"kubernetes.io/projected/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-kube-api-access-4fmdl\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301701    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301716    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcfb74ac-619a-419e-8284-25b3f1a061b9-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301730    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301785    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301798    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.301847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.301811    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c0a8957-f185-4646-bf8e-7c03d2fa0da2-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:01.851997 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.851955    6040 generic.go:356] "Generic (PLEG): container finished" podID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" containerID="2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1" exitCode=0
Oct 29 23:35:01.852498 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.852243    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" event={"ID":"2c0a8957-f185-4646-bf8e-7c03d2fa0da2","Type":"ContainerDied","Data":"2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1"}
Oct 29 23:35:01.852498 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.852328    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh" event={"ID":"2c0a8957-f185-4646-bf8e-7c03d2fa0da2","Type":"ContainerDied","Data":"21db2e3452f67da4397c06aa95626cff9954795df81f8a114838d9cfccad44fa"}
Oct 29 23:35:01.852498 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.852353    6040 scope.go:122] "RemoveContainer" containerID="2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1"
Oct 29 23:35:01.852653 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.852592    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"
Oct 29 23:35:01.853835 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.853811    6040 generic.go:356] "Generic (PLEG): container finished" podID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" containerID="9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29" exitCode=0
Oct 29 23:35:01.853958 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.853879    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"
Oct 29 23:35:01.853958 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.853905    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" event={"ID":"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89","Type":"ContainerDied","Data":"9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29"}
Oct 29 23:35:01.854075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.853978    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf" event={"ID":"a5e71c48-e2a9-4c91-bf79-b5c5baef9c89","Type":"ContainerDied","Data":"1c9f5d07b92d863717fe8dbd169c402a1edb7449f3d13c08fae6c6b2fd176931"}
Oct 29 23:35:01.855417 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.855390    6040 generic.go:356] "Generic (PLEG): container finished" podID="dcfb74ac-619a-419e-8284-25b3f1a061b9" containerID="a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496" exitCode=0
Oct 29 23:35:01.855516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.855440    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"
Oct 29 23:35:01.855516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.855467    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" event={"ID":"dcfb74ac-619a-419e-8284-25b3f1a061b9","Type":"ContainerDied","Data":"a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496"}
Oct 29 23:35:01.855516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.855512    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5" event={"ID":"dcfb74ac-619a-419e-8284-25b3f1a061b9","Type":"ContainerDied","Data":"cec9318beac1776644af158ecbd78b8db506122e2deef0b6b75e1b34f9cba75e"}
Oct 29 23:35:01.857204 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.857184    6040 generic.go:356] "Generic (PLEG): container finished" podID="4a8fbb40-37c6-44ef-8339-5bd0187e687b" containerID="24d890d55c6df4f05240553cba5f7478f883c6c321f156b7b0862e83fb924cca" exitCode=0
Oct 29 23:35:01.857306 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.857238    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g" event={"ID":"4a8fbb40-37c6-44ef-8339-5bd0187e687b","Type":"ContainerDied","Data":"24d890d55c6df4f05240553cba5f7478f883c6c321f156b7b0862e83fb924cca"}
Oct 29 23:35:01.867872 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.867846    6040 scope.go:122] "RemoveContainer" containerID="2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1"
Oct 29 23:35:01.868228 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:01.868207    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1\": container with ID starting with 2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1 not found: ID does not exist" containerID="2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1"
Oct 29 23:35:01.868316 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.868238    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1"} err="failed to get container status \"2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1\": rpc error: code = NotFound desc = could not find container \"2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1\": container with ID starting with 2b602dbb3cfc91c9e08881ab94d8ed5c1c155db4b777c179c17a9f42a0acf4d1 not found: ID does not exist"
Oct 29 23:35:01.868316 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.868255    6040 scope.go:122] "RemoveContainer" containerID="9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29"
Oct 29 23:35:01.877967 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.877948    6040 scope.go:122] "RemoveContainer" containerID="9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29"
Oct 29 23:35:01.878244 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:01.878227    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29\": container with ID starting with 9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29 not found: ID does not exist" containerID="9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29"
Oct 29 23:35:01.878288 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.878252    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29"} err="failed to get container status \"9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29\": rpc error: code = NotFound desc = could not find container \"9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29\": container with ID starting with 9930de5a6587ca45012895264f51ebe49ed31cec29c3d8a4663409e91acb6d29 not found: ID does not exist"
Oct 29 23:35:01.878288 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.878268    6040 scope.go:122] "RemoveContainer" containerID="a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496"
Oct 29 23:35:01.887193 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.887169    6040 scope.go:122] "RemoveContainer" containerID="a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496"
Oct 29 23:35:01.887505 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:01.887485    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496\": container with ID starting with a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496 not found: ID does not exist" containerID="a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496"
Oct 29 23:35:01.887585 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.887514    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496"} err="failed to get container status \"a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496\": rpc error: code = NotFound desc = could not find container \"a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496\": container with ID starting with a6fb9cc410e9b7ed5fbed987849cbd2b9b238d854ccf5475afe300780ab26496 not found: ID does not exist"
Oct 29 23:35:01.981518 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.981490    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"]
Oct 29 23:35:01.985720 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:01.985687    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-b7qvf"]
Oct 29 23:35:02.019356 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:02.019322    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"]
Oct 29 23:35:02.023413 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:02.023380    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wphqh"]
Oct 29 23:35:02.064366 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:02.064317    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"]
Oct 29 23:35:02.071405 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:02.071364    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-wpft5"]
Oct 29 23:35:02.431207 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:02.431170    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" path="/var/lib/kubelet/pods/2c0a8957-f185-4646-bf8e-7c03d2fa0da2/volumes"
Oct 29 23:35:02.431892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:02.431869    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" path="/var/lib/kubelet/pods/a5e71c48-e2a9-4c91-bf79-b5c5baef9c89/volumes"
Oct 29 23:35:02.432489 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:02.432468    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dcfb74ac-619a-419e-8284-25b3f1a061b9" path="/var/lib/kubelet/pods/dcfb74ac-619a-419e-8284-25b3f1a061b9/volumes"
Oct 29 23:35:03.016354 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.016324    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:35:03.122501 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.122416    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a8fbb40-37c6-44ef-8339-5bd0187e687b-kube-api-access-jcm2k\" (UniqueName: \"kubernetes.io/projected/4a8fbb40-37c6-44ef-8339-5bd0187e687b-kube-api-access-jcm2k\") pod \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") "
Oct 29 23:35:03.122501 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.122471    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-openshift-ca\") pod \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") "
Oct 29 23:35:03.122732 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.122539    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-xks-ca\") pod \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\" (UID: \"4a8fbb40-37c6-44ef-8339-5bd0187e687b\") "
Oct 29 23:35:03.122925 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.122892    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-openshift-ca" pod "4a8fbb40-37c6-44ef-8339-5bd0187e687b" (UID: "4a8fbb40-37c6-44ef-8339-5bd0187e687b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:03.123182 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.123155    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:03.124539 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.124517    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-xks-ca" pod "4a8fbb40-37c6-44ef-8339-5bd0187e687b" (UID: "4a8fbb40-37c6-44ef-8339-5bd0187e687b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:35:03.124633 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.124616    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a8fbb40-37c6-44ef-8339-5bd0187e687b-kube-api-access-jcm2k" pod "4a8fbb40-37c6-44ef-8339-5bd0187e687b" (UID: "4a8fbb40-37c6-44ef-8339-5bd0187e687b"). InnerVolumeSpecName "kube-api-access-jcm2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:03.224819 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.224784    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcm2k\" (UniqueName: \"kubernetes.io/projected/4a8fbb40-37c6-44ef-8339-5bd0187e687b-kube-api-access-jcm2k\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:03.224819 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.224813    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4a8fbb40-37c6-44ef-8339-5bd0187e687b-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:03.577256 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.577224    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"]
Oct 29 23:35:03.876999 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.876910    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g" event={"ID":"4a8fbb40-37c6-44ef-8339-5bd0187e687b","Type":"ContainerDied","Data":"7519c98d3c884bf2dffcb4b5c265672877a8fa955f01b58f8f12835ea238b226"}
Oct 29 23:35:03.876999 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.876979    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7519c98d3c884bf2dffcb4b5c265672877a8fa955f01b58f8f12835ea238b226"
Oct 29 23:35:03.877175 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:03.877048    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"
Oct 29 23:35:04.058579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.058540    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" podUID="680505f8-f92f-4225-b61f-5b323c23b236" containerName="main" containerID="cri-o://b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646" gracePeriod=45
Oct 29 23:35:04.069771 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.069712    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main" containerID="cri-o://538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440" gracePeriod=45
Oct 29 23:35:04.406523 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.406489    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:35:04.409673 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.409653    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:35:04.440649 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.440617    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a37b6f4c-bacc-4584-bbf2-b2414fa85897-kube-api-access-zr2nt\" (UniqueName: \"kubernetes.io/projected/a37b6f4c-bacc-4584-bbf2-b2414fa85897-kube-api-access-zr2nt\") pod \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") "
Oct 29 23:35:04.440820 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.440657    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-dshm\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-dshm\") pod \"680505f8-f92f-4225-b61f-5b323c23b236\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") "
Oct 29 23:35:04.440820 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.440697    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-model-cache\") pod \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") "
Oct 29 23:35:04.440820 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.440728    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/680505f8-f92f-4225-b61f-5b323c23b236-tls-certs\" (UniqueName: \"kubernetes.io/secret/680505f8-f92f-4225-b61f-5b323c23b236-tls-certs\") pod \"680505f8-f92f-4225-b61f-5b323c23b236\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") "
Oct 29 23:35:04.440820 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.440789    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/680505f8-f92f-4225-b61f-5b323c23b236-kube-api-access-vskzw\" (UniqueName: \"kubernetes.io/projected/680505f8-f92f-4225-b61f-5b323c23b236-kube-api-access-vskzw\") pod \"680505f8-f92f-4225-b61f-5b323c23b236\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") "
Oct 29 23:35:04.441054 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.440866    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-home\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-home\") pod \"680505f8-f92f-4225-b61f-5b323c23b236\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") "
Oct 29 23:35:04.441054 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.440930    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tls-certs\" (UniqueName: \"kubernetes.io/secret/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tls-certs\") pod \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") "
Oct 29 23:35:04.441054 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.440960    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-tmp-dir\") pod \"680505f8-f92f-4225-b61f-5b323c23b236\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") "
Oct 29 23:35:04.441054 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.440995    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-model-cache\") pod \"680505f8-f92f-4225-b61f-5b323c23b236\" (UID: \"680505f8-f92f-4225-b61f-5b323c23b236\") "
Oct 29 23:35:04.441054 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.441052    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-dshm\") pod \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") "
Oct 29 23:35:04.441298 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.441077    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-home\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-home\") pod \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") "
Oct 29 23:35:04.441298 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.441198    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tmp-dir\") pod \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\" (UID: \"a37b6f4c-bacc-4584-bbf2-b2414fa85897\") "
Oct 29 23:35:04.441298 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.441264    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-model-cache" pod "a37b6f4c-bacc-4584-bbf2-b2414fa85897" (UID: "a37b6f4c-bacc-4584-bbf2-b2414fa85897"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:04.441526 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.441505    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-tmp-dir" pod "680505f8-f92f-4225-b61f-5b323c23b236" (UID: "680505f8-f92f-4225-b61f-5b323c23b236"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:04.441731 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.441708    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tmp-dir" pod "a37b6f4c-bacc-4584-bbf2-b2414fa85897" (UID: "a37b6f4c-bacc-4584-bbf2-b2414fa85897"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:04.441944 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.441922    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-home" pod "680505f8-f92f-4225-b61f-5b323c23b236" (UID: "680505f8-f92f-4225-b61f-5b323c23b236"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:04.442006 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.441975    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.442006 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.441999    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.442107 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.442014    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.442229 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.442207    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-model-cache" pod "680505f8-f92f-4225-b61f-5b323c23b236" (UID: "680505f8-f92f-4225-b61f-5b323c23b236"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:04.442408 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.442383    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-home" pod "a37b6f4c-bacc-4584-bbf2-b2414fa85897" (UID: "a37b6f4c-bacc-4584-bbf2-b2414fa85897"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:04.443568 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.443540    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a37b6f4c-bacc-4584-bbf2-b2414fa85897-kube-api-access-zr2nt" pod "a37b6f4c-bacc-4584-bbf2-b2414fa85897" (UID: "a37b6f4c-bacc-4584-bbf2-b2414fa85897"). InnerVolumeSpecName "kube-api-access-zr2nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:04.444510 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.444484    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tls-certs" pod "a37b6f4c-bacc-4584-bbf2-b2414fa85897" (UID: "a37b6f4c-bacc-4584-bbf2-b2414fa85897"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:35:04.444850 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.444820    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-dshm" pod "680505f8-f92f-4225-b61f-5b323c23b236" (UID: "680505f8-f92f-4225-b61f-5b323c23b236"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:04.444956 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.444831    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-dshm" pod "a37b6f4c-bacc-4584-bbf2-b2414fa85897" (UID: "a37b6f4c-bacc-4584-bbf2-b2414fa85897"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:04.445184 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.445161    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/680505f8-f92f-4225-b61f-5b323c23b236-tls-certs" pod "680505f8-f92f-4225-b61f-5b323c23b236" (UID: "680505f8-f92f-4225-b61f-5b323c23b236"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:35:04.445294 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.445192    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/680505f8-f92f-4225-b61f-5b323c23b236-kube-api-access-vskzw" pod "680505f8-f92f-4225-b61f-5b323c23b236" (UID: "680505f8-f92f-4225-b61f-5b323c23b236"). InnerVolumeSpecName "kube-api-access-vskzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:04.543409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.543376    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.543409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.543400    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37b6f4c-bacc-4584-bbf2-b2414fa85897-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.543409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.543410    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr2nt\" (UniqueName: \"kubernetes.io/projected/a37b6f4c-bacc-4584-bbf2-b2414fa85897-kube-api-access-zr2nt\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.543409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.543420    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.543797 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.543429    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/680505f8-f92f-4225-b61f-5b323c23b236-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.543797 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.543437    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vskzw\" (UniqueName: \"kubernetes.io/projected/680505f8-f92f-4225-b61f-5b323c23b236-kube-api-access-vskzw\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.543797 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.543445    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.543797 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.543453    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37b6f4c-bacc-4584-bbf2-b2414fa85897-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.543797 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.543461    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/680505f8-f92f-4225-b61f-5b323c23b236-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:04.785842 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.785757    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:35:04.884383 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.884347    6040 generic.go:356] "Generic (PLEG): container finished" podID="680505f8-f92f-4225-b61f-5b323c23b236" containerID="b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646" exitCode=0
Oct 29 23:35:04.884550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.884393    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"
Oct 29 23:35:04.884550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.884433    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" event={"ID":"680505f8-f92f-4225-b61f-5b323c23b236","Type":"ContainerDied","Data":"b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646"}
Oct 29 23:35:04.884550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.884476    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq" event={"ID":"680505f8-f92f-4225-b61f-5b323c23b236","Type":"ContainerDied","Data":"efea6f9776a9caa5075b449f6056d8edbc5dc1831ae957d1a37f25c315b53cd4"}
Oct 29 23:35:04.884550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.884493    6040 scope.go:122] "RemoveContainer" containerID="b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646"
Oct 29 23:35:04.886076 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.886050    6040 generic.go:356] "Generic (PLEG): container finished" podID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerID="538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440" exitCode=0
Oct 29 23:35:04.886179 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.886085    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"
Oct 29 23:35:04.886179 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.886131    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" event={"ID":"a37b6f4c-bacc-4584-bbf2-b2414fa85897","Type":"ContainerDied","Data":"538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440"}
Oct 29 23:35:04.886179 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.886170    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94" event={"ID":"a37b6f4c-bacc-4584-bbf2-b2414fa85897","Type":"ContainerDied","Data":"145a925a247962f4a1419435399dc30a197703c2ecadb4f1216a41434e76fa6b"}
Oct 29 23:35:04.895410 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.895389    6040 scope.go:122] "RemoveContainer" containerID="b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646"
Oct 29 23:35:04.895682 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:04.895658    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646\": container with ID starting with b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646 not found: ID does not exist" containerID="b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646"
Oct 29 23:35:04.895766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.895698    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646"} err="failed to get container status \"b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646\": rpc error: code = NotFound desc = could not find container \"b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646\": container with ID starting with b85b625069165142985fe8906ba946c790bc6d15c67504f3bed01c35419c1646 not found: ID does not exist"
Oct 29 23:35:04.895766 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.895723    6040 scope.go:122] "RemoveContainer" containerID="538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440"
Oct 29 23:35:04.904712 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.904695    6040 scope.go:122] "RemoveContainer" containerID="538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440"
Oct 29 23:35:04.904995 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:04.904971    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440\": container with ID starting with 538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440 not found: ID does not exist" containerID="538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440"
Oct 29 23:35:04.905063 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.905002    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440"} err="failed to get container status \"538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440\": rpc error: code = NotFound desc = could not find container \"538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440\": container with ID starting with 538c4ea0552da2a1272d235a57f26b9eca98d9cba1003f67aecf7a69c7dc1440 not found: ID does not exist"
Oct 29 23:35:04.936206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.936180    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"]
Oct 29 23:35:04.940241 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.940220    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/premium-w2-ae1364-kserve-74f4d67488-gws94"]
Oct 29 23:35:04.974341 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.974316    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"]
Oct 29 23:35:04.981898 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:04.981875    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-ae1364/distinct-2-w2-ae1364-kserve-6464c5d99b-zwrjq"]
Oct 29 23:35:06.439153 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:06.439116    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="680505f8-f92f-4225-b61f-5b323c23b236" path="/var/lib/kubelet/pods/680505f8-f92f-4225-b61f-5b323c23b236/volumes"
Oct 29 23:35:06.440801 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:06.440622    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" path="/var/lib/kubelet/pods/a37b6f4c-bacc-4584-bbf2-b2414fa85897/volumes"
Oct 29 23:35:08.462981 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:08.462951    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"]
Oct 29 23:35:08.465551 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:08.463127    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr" podUID="df0c1ec4-67dd-424e-97bf-91e504744d37" containerName="maas-api" containerID="cri-o://36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2" gracePeriod=30
Oct 29 23:35:08.578145 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:08.578087    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh" podUID="7bbff9d3-81f5-4f3f-9be1-afba3a8058ef" containerName="payload-pre-processing" containerID="cri-o://e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377" gracePeriod=25
Oct 29 23:35:09.145971 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.145928    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr" podUID="df0c1ec4-67dd-424e-97bf-91e504744d37" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.46:8443/health\": dial tcp 10.133.0.46:8443: connect: connection refused"
Oct 29 23:35:09.756320 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.756290    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:35:09.818449 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.818369    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-maas-api-tls\") pod \"df0c1ec4-67dd-424e-97bf-91e504744d37\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") "
Oct 29 23:35:09.818640 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.818472    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-metrics-certs\" (UniqueName: \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-metrics-certs\") pod \"df0c1ec4-67dd-424e-97bf-91e504744d37\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") "
Oct 29 23:35:09.818640 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.818570    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df0c1ec4-67dd-424e-97bf-91e504744d37-kube-api-access-5m687\" (UniqueName: \"kubernetes.io/projected/df0c1ec4-67dd-424e-97bf-91e504744d37-kube-api-access-5m687\") pod \"df0c1ec4-67dd-424e-97bf-91e504744d37\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") "
Oct 29 23:35:09.818640 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.818601    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df0c1ec4-67dd-424e-97bf-91e504744d37-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df0c1ec4-67dd-424e-97bf-91e504744d37-openshift-service-ca\") pod \"df0c1ec4-67dd-424e-97bf-91e504744d37\" (UID: \"df0c1ec4-67dd-424e-97bf-91e504744d37\") "
Oct 29 23:35:09.819516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.819419    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df0c1ec4-67dd-424e-97bf-91e504744d37-openshift-service-ca" pod "df0c1ec4-67dd-424e-97bf-91e504744d37" (UID: "df0c1ec4-67dd-424e-97bf-91e504744d37"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:09.822772 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.821648    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-maas-api-tls" pod "df0c1ec4-67dd-424e-97bf-91e504744d37" (UID: "df0c1ec4-67dd-424e-97bf-91e504744d37"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:35:09.822772 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.821811    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-metrics-certs" pod "df0c1ec4-67dd-424e-97bf-91e504744d37" (UID: "df0c1ec4-67dd-424e-97bf-91e504744d37"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:35:09.824199 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.824169    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0c1ec4-67dd-424e-97bf-91e504744d37-kube-api-access-5m687" pod "df0c1ec4-67dd-424e-97bf-91e504744d37" (UID: "df0c1ec4-67dd-424e-97bf-91e504744d37"). InnerVolumeSpecName "kube-api-access-5m687". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:09.920951 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.920907    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-maas-api-tls\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:09.920951 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.920941    6040 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df0c1ec4-67dd-424e-97bf-91e504744d37-metrics-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:09.920951 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.920956    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m687\" (UniqueName: \"kubernetes.io/projected/df0c1ec4-67dd-424e-97bf-91e504744d37-kube-api-access-5m687\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:09.921202 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.920970    6040 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df0c1ec4-67dd-424e-97bf-91e504744d37-openshift-service-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:09.948869 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.948825    6040 generic.go:356] "Generic (PLEG): container finished" podID="df0c1ec4-67dd-424e-97bf-91e504744d37" containerID="36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2" exitCode=0
Oct 29 23:35:09.949056 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.948894    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"
Oct 29 23:35:09.949056 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.948922    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr" event={"ID":"df0c1ec4-67dd-424e-97bf-91e504744d37","Type":"ContainerDied","Data":"36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2"}
Oct 29 23:35:09.949056 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.949021    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr" event={"ID":"df0c1ec4-67dd-424e-97bf-91e504744d37","Type":"ContainerDied","Data":"d606f844fb097c7f8acf047ff2ffba2134eaa05aedfb96aae83748ac24835114"}
Oct 29 23:35:09.949056 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.949056    6040 scope.go:122] "RemoveContainer" containerID="36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2"
Oct 29 23:35:09.961579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.961555    6040 scope.go:122] "RemoveContainer" containerID="36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2"
Oct 29 23:35:09.961956 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:09.961928    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2\": container with ID starting with 36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2 not found: ID does not exist" containerID="36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2"
Oct 29 23:35:09.962068 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:09.961959    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2"} err="failed to get container status \"36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2\": rpc error: code = NotFound desc = could not find container \"36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2\": container with ID starting with 36722ca006e72ca429538ffd47f8d0793c758322787c95b16845ebe9b82679d2 not found: ID does not exist"
Oct 29 23:35:10.016550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:10.016482    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"]
Oct 29 23:35:10.025181 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:10.025152    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ae1364-58c86594cd-c6dgr"]
Oct 29 23:35:10.063772 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:10.057472    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"]
Oct 29 23:35:10.078894 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:10.078811    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-00b7cd06-lq78c"]
Oct 29 23:35:10.095457 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:10.095421    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh" podUID="7bbff9d3-81f5-4f3f-9be1-afba3a8058ef" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 29 23:35:10.464763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:10.464397    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cdbe482f-fa7e-4fe0-bc76-a01df162bf4e" path="/var/lib/kubelet/pods/cdbe482f-fa7e-4fe0-bc76-a01df162bf4e/volumes"
Oct 29 23:35:10.468758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:10.465058    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df0c1ec4-67dd-424e-97bf-91e504744d37" path="/var/lib/kubelet/pods/df0c1ec4-67dd-424e-97bf-91e504744d37/volumes"
Oct 29 23:35:12.854856 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:12.854815    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" containerID="cri-o://348dbc4f806b3acb06e429977e5646b140b926736b523feded21c29078f52200" gracePeriod=45
Oct 29 23:35:12.971011 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:12.970979    6040 generic.go:356] "Generic (PLEG): container finished" podID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerID="348dbc4f806b3acb06e429977e5646b140b926736b523feded21c29078f52200" exitCode=0
Oct 29 23:35:12.971158 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:12.971021    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" event={"ID":"d3b29afd-299b-4dfc-b78f-73981c1c343a","Type":"ContainerDied","Data":"348dbc4f806b3acb06e429977e5646b140b926736b523feded21c29078f52200"}
Oct 29 23:35:12.992409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:12.992371    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 29 23:35:13.115635 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.115578    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:35:13.271358 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.271325    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-tmp-dir\") pod \"d3b29afd-299b-4dfc-b78f-73981c1c343a\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") "
Oct 29 23:35:13.271530 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.271380    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3b29afd-299b-4dfc-b78f-73981c1c343a-kube-api-access-6rh75\" (UniqueName: \"kubernetes.io/projected/d3b29afd-299b-4dfc-b78f-73981c1c343a-kube-api-access-6rh75\") pod \"d3b29afd-299b-4dfc-b78f-73981c1c343a\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") "
Oct 29 23:35:13.271530 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.271436    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3b29afd-299b-4dfc-b78f-73981c1c343a-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b29afd-299b-4dfc-b78f-73981c1c343a-tls-certs\") pod \"d3b29afd-299b-4dfc-b78f-73981c1c343a\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") "
Oct 29 23:35:13.271666 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.271541    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-model-cache\") pod \"d3b29afd-299b-4dfc-b78f-73981c1c343a\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") "
Oct 29 23:35:13.271666 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.271577    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-dshm\") pod \"d3b29afd-299b-4dfc-b78f-73981c1c343a\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") "
Oct 29 23:35:13.271666 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.271623    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-home\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-home\") pod \"d3b29afd-299b-4dfc-b78f-73981c1c343a\" (UID: \"d3b29afd-299b-4dfc-b78f-73981c1c343a\") "
Oct 29 23:35:13.271666 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.271629    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-tmp-dir" pod "d3b29afd-299b-4dfc-b78f-73981c1c343a" (UID: "d3b29afd-299b-4dfc-b78f-73981c1c343a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:13.271905 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.271848    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-model-cache" pod "d3b29afd-299b-4dfc-b78f-73981c1c343a" (UID: "d3b29afd-299b-4dfc-b78f-73981c1c343a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:13.271957 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.271940    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-home" pod "d3b29afd-299b-4dfc-b78f-73981c1c343a" (UID: "d3b29afd-299b-4dfc-b78f-73981c1c343a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:13.272116 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.272098    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:13.272183 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.272116    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:13.272183 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.272127    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:13.273891 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.273866    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3b29afd-299b-4dfc-b78f-73981c1c343a-tls-certs" pod "d3b29afd-299b-4dfc-b78f-73981c1c343a" (UID: "d3b29afd-299b-4dfc-b78f-73981c1c343a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:35:13.273998 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.273907    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3b29afd-299b-4dfc-b78f-73981c1c343a-kube-api-access-6rh75" pod "d3b29afd-299b-4dfc-b78f-73981c1c343a" (UID: "d3b29afd-299b-4dfc-b78f-73981c1c343a"). InnerVolumeSpecName "kube-api-access-6rh75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:13.273998 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.273932    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-dshm" pod "d3b29afd-299b-4dfc-b78f-73981c1c343a" (UID: "d3b29afd-299b-4dfc-b78f-73981c1c343a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:13.373180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.373097    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rh75\" (UniqueName: \"kubernetes.io/projected/d3b29afd-299b-4dfc-b78f-73981c1c343a-kube-api-access-6rh75\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:13.373180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.373122    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b29afd-299b-4dfc-b78f-73981c1c343a-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:13.373180 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.373132    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b29afd-299b-4dfc-b78f-73981c1c343a-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:13.980278 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.980248    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"
Oct 29 23:35:13.980709 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.980235    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff" event={"ID":"d3b29afd-299b-4dfc-b78f-73981c1c343a","Type":"ContainerDied","Data":"9ddbcfeb41de56e90420c04de61976f546f0b8f76bf1d0f5bdebfee51d059a22"}
Oct 29 23:35:13.980709 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:13.980380    6040 scope.go:122] "RemoveContainer" containerID="348dbc4f806b3acb06e429977e5646b140b926736b523feded21c29078f52200"
Oct 29 23:35:14.033999 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:14.033970    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"]
Oct 29 23:35:14.045551 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:14.045530    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/simulator-w6-ae1784-kserve-7db7fbc88c-9rwff"]
Oct 29 23:35:14.425521 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:14.425489    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" path="/var/lib/kubelet/pods/d3b29afd-299b-4dfc-b78f-73981c1c343a/volumes"
Oct 29 23:35:15.034866 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:15.034834    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"]
Oct 29 23:35:15.046005 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:15.045981    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-347c0d86-bwnht"]
Oct 29 23:35:16.178052 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:16.178020    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"]
Oct 29 23:35:16.195612 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:16.195569    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"]
Oct 29 23:35:16.229354 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:16.227846    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"]
Oct 29 23:35:16.259296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:16.259265    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"]
Oct 29 23:35:16.444819 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:16.444710    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3895e845-9690-47db-89a0-0c716657236a" path="/var/lib/kubelet/pods/3895e845-9690-47db-89a0-0c716657236a/volumes"
Oct 29 23:35:17.495299 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:17.495264    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"]
Oct 29 23:35:17.495852 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:17.495494    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg" podUID="49d459ae-07ae-4c53-aae6-934c25119c6c" containerName="istio-proxy" containerID="cri-o://7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18" gracePeriod=30
Oct 29 23:35:17.509998 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:17.509963    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"]
Oct 29 23:35:17.510271 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:17.510245    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy" containerID="cri-o://cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653" gracePeriod=30
Oct 29 23:35:17.520747 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:17.520704    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"]
Oct 29 23:35:17.520942 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:17.520922    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" podUID="f45f800d-3877-415d-b832-92956ed12f88" containerName="istio-proxy" containerID="cri-o://18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3" gracePeriod=30
Oct 29 23:35:18.753025 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:18.752986    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 29 23:35:18.753418 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:18.753051    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" podUID="f45f800d-3877-415d-b832-92956ed12f88" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:35:20.095543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:20.095508    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh" podUID="7bbff9d3-81f5-4f3f-9be1-afba3a8058ef" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 29 23:35:22.146645 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.146604    6040 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 29 23:35:22.147048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.146669    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:35:22.203886 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.203851    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:35:22.777488 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.777429    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:35:22.781731 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.781703    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f45f800d-3877-415d-b832-92956ed12f88-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45f800d-3877-415d-b832-92956ed12f88-istiod-ca-cert\") pod \"f45f800d-3877-415d-b832-92956ed12f88\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") "
Oct 29 23:35:22.781879 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.781795    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-kube-api-access-s9br4\" (UniqueName: \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-kube-api-access-s9br4\") pod \"f45f800d-3877-415d-b832-92956ed12f88\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") "
Oct 29 23:35:22.781879 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.781836    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f45f800d-3877-415d-b832-92956ed12f88-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45f800d-3877-415d-b832-92956ed12f88-istio-podinfo\") pod \"f45f800d-3877-415d-b832-92956ed12f88\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") "
Oct 29 23:35:22.781879 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.781869    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-socket\") pod \"f45f800d-3877-415d-b832-92956ed12f88\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") "
Oct 29 23:35:22.781987 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.781920    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-credential-socket\") pod \"f45f800d-3877-415d-b832-92956ed12f88\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") "
Oct 29 23:35:22.782079 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.782060    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-envoy\") pod \"f45f800d-3877-415d-b832-92956ed12f88\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") "
Oct 29 23:35:22.782131 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.782117    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-data\") pod \"f45f800d-3877-415d-b832-92956ed12f88\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") "
Oct 29 23:35:22.782178 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.782160    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-certs\") pod \"f45f800d-3877-415d-b832-92956ed12f88\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") "
Oct 29 23:35:22.782844 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.782205    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-istio-token\" (UniqueName: \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-istio-token\") pod \"f45f800d-3877-415d-b832-92956ed12f88\" (UID: \"f45f800d-3877-415d-b832-92956ed12f88\") "
Oct 29 23:35:22.782844 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.782416    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-socket" pod "f45f800d-3877-415d-b832-92956ed12f88" (UID: "f45f800d-3877-415d-b832-92956ed12f88"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.782844 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.782750    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.783028 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.782975    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-data" pod "f45f800d-3877-415d-b832-92956ed12f88" (UID: "f45f800d-3877-415d-b832-92956ed12f88"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.783164 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.783136    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-credential-socket" pod "f45f800d-3877-415d-b832-92956ed12f88" (UID: "f45f800d-3877-415d-b832-92956ed12f88"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.783431 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.783406    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-certs" pod "f45f800d-3877-415d-b832-92956ed12f88" (UID: "f45f800d-3877-415d-b832-92956ed12f88"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.783960 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.783728    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f45f800d-3877-415d-b832-92956ed12f88-istiod-ca-cert" pod "f45f800d-3877-415d-b832-92956ed12f88" (UID: "f45f800d-3877-415d-b832-92956ed12f88"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:22.785231 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.785006    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f45f800d-3877-415d-b832-92956ed12f88-istio-podinfo" pod "f45f800d-3877-415d-b832-92956ed12f88" (UID: "f45f800d-3877-415d-b832-92956ed12f88"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:22.785231 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.785037    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-istio-token" pod "f45f800d-3877-415d-b832-92956ed12f88" (UID: "f45f800d-3877-415d-b832-92956ed12f88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:22.786377 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.786347    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-envoy" pod "f45f800d-3877-415d-b832-92956ed12f88" (UID: "f45f800d-3877-415d-b832-92956ed12f88"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.787064 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.787046    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-kube-api-access-s9br4" pod "f45f800d-3877-415d-b832-92956ed12f88" (UID: "f45f800d-3877-415d-b832-92956ed12f88"). InnerVolumeSpecName "kube-api-access-s9br4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:22.825913 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.825890    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:35:22.848364 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.848329    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883317    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-token\" (UniqueName: \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-token\") pod \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883373    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-podinfo\") pod \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883409    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-certs\") pod \"49d459ae-07ae-4c53-aae6-934c25119c6c\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883460    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-data\") pod \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883511    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-credential-socket\") pod \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883553    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-envoy\") pod \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883597    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-certs\") pod \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883629    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-kube-api-access-jpwwg\" (UniqueName: \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-kube-api-access-jpwwg\") pod \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883674    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-kube-api-access-k774h\" (UniqueName: \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-kube-api-access-k774h\") pod \"49d459ae-07ae-4c53-aae6-934c25119c6c\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883717    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-token\" (UniqueName: \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-token\") pod \"49d459ae-07ae-4c53-aae6-934c25119c6c\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883790    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-socket\") pod \"49d459ae-07ae-4c53-aae6-934c25119c6c\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883826    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-envoy\") pod \"49d459ae-07ae-4c53-aae6-934c25119c6c\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883854    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-podinfo\") pod \"49d459ae-07ae-4c53-aae6-934c25119c6c\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883882    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istiod-ca-cert\") pod \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883922    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-socket\") pod \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\" (UID: \"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.883960    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49d459ae-07ae-4c53-aae6-934c25119c6c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49d459ae-07ae-4c53-aae6-934c25119c6c-istiod-ca-cert\") pod \"49d459ae-07ae-4c53-aae6-934c25119c6c\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884011    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-credential-socket" pod "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" (UID: "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884042    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-credential-socket\") pod \"49d459ae-07ae-4c53-aae6-934c25119c6c\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884068    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-data\") pod \"49d459ae-07ae-4c53-aae6-934c25119c6c\" (UID: \"49d459ae-07ae-4c53-aae6-934c25119c6c\") "
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884531    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9br4\" (UniqueName: \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-kube-api-access-s9br4\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884549    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45f800d-3877-415d-b832-92956ed12f88-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884558    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884567    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884576    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884585    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884593    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45f800d-3877-415d-b832-92956ed12f88-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884600    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45f800d-3877-415d-b832-92956ed12f88-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.884780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.884608    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45f800d-3877-415d-b832-92956ed12f88-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.887258 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.886350    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-certs" pod "49d459ae-07ae-4c53-aae6-934c25119c6c" (UID: "49d459ae-07ae-4c53-aae6-934c25119c6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.887258 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.886571    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-data" pod "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" (UID: "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.894532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.894010    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49d459ae-07ae-4c53-aae6-934c25119c6c-istiod-ca-cert" pod "49d459ae-07ae-4c53-aae6-934c25119c6c" (UID: "49d459ae-07ae-4c53-aae6-934c25119c6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:22.894532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.894199    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-socket" pod "49d459ae-07ae-4c53-aae6-934c25119c6c" (UID: "49d459ae-07ae-4c53-aae6-934c25119c6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.894532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.894385    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-certs" pod "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" (UID: "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.894532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.894413    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-socket" pod "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" (UID: "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.894532 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.894526    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istiod-ca-cert" pod "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" (UID: "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:22.894913 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.894574    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-credential-socket" pod "49d459ae-07ae-4c53-aae6-934c25119c6c" (UID: "49d459ae-07ae-4c53-aae6-934c25119c6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.897332 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.895570    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-data" pod "49d459ae-07ae-4c53-aae6-934c25119c6c" (UID: "49d459ae-07ae-4c53-aae6-934c25119c6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.897709 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.897675    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-token" pod "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" (UID: "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:22.902239 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.902204    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-kube-api-access-jpwwg" pod "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" (UID: "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b"). InnerVolumeSpecName "kube-api-access-jpwwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:22.906686 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.906652    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-token" pod "49d459ae-07ae-4c53-aae6-934c25119c6c" (UID: "49d459ae-07ae-4c53-aae6-934c25119c6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:22.932768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.931367    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-kube-api-access-k774h" pod "49d459ae-07ae-4c53-aae6-934c25119c6c" (UID: "49d459ae-07ae-4c53-aae6-934c25119c6c"). InnerVolumeSpecName "kube-api-access-k774h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:22.932768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.931505    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-envoy" pod "49d459ae-07ae-4c53-aae6-934c25119c6c" (UID: "49d459ae-07ae-4c53-aae6-934c25119c6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.932768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.931771    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-envoy" pod "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" (UID: "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:22.932768 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.931874    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-podinfo" pod "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" (UID: "7c3d6233-ff6f-4821-861f-9dd0eacbbe6b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:22.934026 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.933995    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-podinfo" pod "49d459ae-07ae-4c53-aae6-934c25119c6c" (UID: "49d459ae-07ae-4c53-aae6-934c25119c6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.985873    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.985911    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.985926    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpwwg\" (UniqueName: \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-kube-api-access-jpwwg\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.985943    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k774h\" (UniqueName: \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-kube-api-access-k774h\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.985957    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.985970    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.985983    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.985996    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.986008    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.986021    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.986034    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49d459ae-07ae-4c53-aae6-934c25119c6c-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.986048    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.986060    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.986073    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.986085    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.986097    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49d459ae-07ae-4c53-aae6-934c25119c6c-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:22.986135 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:22.986109    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:23.039119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.039036    6040 generic.go:356] "Generic (PLEG): container finished" podID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerID="cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653" exitCode=0
Oct 29 23:35:23.039119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.039100    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"
Oct 29 23:35:23.039342 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.039130    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" event={"ID":"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b","Type":"ContainerDied","Data":"cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653"}
Oct 29 23:35:23.039342 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.039204    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7" event={"ID":"7c3d6233-ff6f-4821-861f-9dd0eacbbe6b","Type":"ContainerDied","Data":"51a296b134afe0a0d5a889f797f9870c4d1f30f371ced7e983ec760a5ca4e419"}
Oct 29 23:35:23.039342 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.039251    6040 scope.go:122] "RemoveContainer" containerID="cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653"
Oct 29 23:35:23.046272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.046248    6040 generic.go:356] "Generic (PLEG): container finished" podID="f45f800d-3877-415d-b832-92956ed12f88" containerID="18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3" exitCode=0
Oct 29 23:35:23.046399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.046304    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"
Oct 29 23:35:23.046399 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.046370    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" event={"ID":"f45f800d-3877-415d-b832-92956ed12f88","Type":"ContainerDied","Data":"18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3"}
Oct 29 23:35:23.046550 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.046429    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8" event={"ID":"f45f800d-3877-415d-b832-92956ed12f88","Type":"ContainerDied","Data":"7357ee653daa4cec10663757f2354721d9c2733710b1b5697d950a28f3f4ccb3"}
Oct 29 23:35:23.053027 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.052998    6040 generic.go:356] "Generic (PLEG): container finished" podID="49d459ae-07ae-4c53-aae6-934c25119c6c" containerID="7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18" exitCode=0
Oct 29 23:35:23.053138 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.053047    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg" event={"ID":"49d459ae-07ae-4c53-aae6-934c25119c6c","Type":"ContainerDied","Data":"7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18"}
Oct 29 23:35:23.053138 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.053088    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg" event={"ID":"49d459ae-07ae-4c53-aae6-934c25119c6c","Type":"ContainerDied","Data":"ad64feb6b1a6b3bcef3d3570444e46907f90f93651059470cbd4659f8ebaeed1"}
Oct 29 23:35:23.053269 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.053215    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"
Oct 29 23:35:23.063334 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.063310    6040 scope.go:122] "RemoveContainer" containerID="cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653"
Oct 29 23:35:23.067053 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:23.066120    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653\": container with ID starting with cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653 not found: ID does not exist" containerID="cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653"
Oct 29 23:35:23.067053 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.066159    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653"} err="failed to get container status \"cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653\": rpc error: code = NotFound desc = could not find container \"cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653\": container with ID starting with cd56dd35c0191fa99715460a2fbd0ba0d67818bc0940d3ba31068a3d3c157653 not found: ID does not exist"
Oct 29 23:35:23.067053 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.066183    6040 scope.go:122] "RemoveContainer" containerID="18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3"
Oct 29 23:35:23.091829 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.091802    6040 scope.go:122] "RemoveContainer" containerID="18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3"
Oct 29 23:35:23.092470 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:23.092342    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3\": container with ID starting with 18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3 not found: ID does not exist" containerID="18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3"
Oct 29 23:35:23.092470 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.092380    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3"} err="failed to get container status \"18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3\": rpc error: code = NotFound desc = could not find container \"18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3\": container with ID starting with 18ef582b9b094e86140e2bc1e3897ccdeb1446978dcd763bbf0f67765d623de3 not found: ID does not exist"
Oct 29 23:35:23.092470 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.092403    6040 scope.go:122] "RemoveContainer" containerID="7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18"
Oct 29 23:35:23.106993 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.106970    6040 scope.go:122] "RemoveContainer" containerID="7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18"
Oct 29 23:35:23.107443 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:23.107361    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18\": container with ID starting with 7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18 not found: ID does not exist" containerID="7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18"
Oct 29 23:35:23.107443 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.107400    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18"} err="failed to get container status \"7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18\": rpc error: code = NotFound desc = could not find container \"7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18\": container with ID starting with 7a42e75e55f68c6321d60430aeecee0b937295d386009a0424eac60b78897d18 not found: ID does not exist"
Oct 29 23:35:23.138137 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.138097    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"]
Oct 29 23:35:23.146765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.146711    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-dnld7"]
Oct 29 23:35:23.202459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.198027    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"]
Oct 29 23:35:23.202459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.202375    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-rpfmg"]
Oct 29 23:35:23.237296 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.237260    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"]
Oct 29 23:35:23.250718 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.250677    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-99vl8"]
Oct 29 23:35:23.738662 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.738624    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"]
Oct 29 23:35:23.739578 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.739551    6040 cpu_manager.go:414] "containerMap: removing container" podUID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" containerName="istio-proxy"
Oct 29 23:35:23.739716 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.739701    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" containerName="istio-proxy"
Oct 29 23:35:23.739910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.739887    6040 cpu_manager.go:414] "containerMap: removing container" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main"
Oct 29 23:35:23.740017 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740005    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main"
Oct 29 23:35:23.740106 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740096    6040 cpu_manager.go:414] "containerMap: removing container" podUID="f45f800d-3877-415d-b832-92956ed12f88" containerName="istio-proxy"
Oct 29 23:35:23.740187 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740175    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f45f800d-3877-415d-b832-92956ed12f88" containerName="istio-proxy"
Oct 29 23:35:23.740272 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740261    6040 cpu_manager.go:414] "containerMap: removing container" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy"
Oct 29 23:35:23.740359 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740346    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy"
Oct 29 23:35:23.740450 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740440    6040 cpu_manager.go:414] "containerMap: removing container" podUID="dcfb74ac-619a-419e-8284-25b3f1a061b9" containerName="istio-proxy"
Oct 29 23:35:23.740529 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740519    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcfb74ac-619a-419e-8284-25b3f1a061b9" containerName="istio-proxy"
Oct 29 23:35:23.740610 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740601    6040 cpu_manager.go:414] "containerMap: removing container" podUID="4a8fbb40-37c6-44ef-8339-5bd0187e687b" containerName="revoke-keys"
Oct 29 23:35:23.740679 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740669    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a8fbb40-37c6-44ef-8339-5bd0187e687b" containerName="revoke-keys"
Oct 29 23:35:23.740800 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740777    6040 cpu_manager.go:414] "containerMap: removing container" podUID="49d459ae-07ae-4c53-aae6-934c25119c6c" containerName="istio-proxy"
Oct 29 23:35:23.740917 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740903    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49d459ae-07ae-4c53-aae6-934c25119c6c" containerName="istio-proxy"
Oct 29 23:35:23.741008 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.740997    6040 cpu_manager.go:414] "containerMap: removing container" podUID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" containerName="istio-proxy"
Oct 29 23:35:23.741092 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.741081    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" containerName="istio-proxy"
Oct 29 23:35:23.741186 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.741175    6040 cpu_manager.go:414] "containerMap: removing container" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main"
Oct 29 23:35:23.741265 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.741254    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main"
Oct 29 23:35:23.741359 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.741347    6040 cpu_manager.go:414] "containerMap: removing container" podUID="df0c1ec4-67dd-424e-97bf-91e504744d37" containerName="maas-api"
Oct 29 23:35:23.741445 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.741434    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df0c1ec4-67dd-424e-97bf-91e504744d37" containerName="maas-api"
Oct 29 23:35:23.741526 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.741515    6040 cpu_manager.go:414] "containerMap: removing container" podUID="680505f8-f92f-4225-b61f-5b323c23b236" containerName="main"
Oct 29 23:35:23.741602 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.741591    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="680505f8-f92f-4225-b61f-5b323c23b236" containerName="main"
Oct 29 23:35:23.741831 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.741814    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c0a8957-f185-4646-bf8e-7c03d2fa0da2" containerName="istio-proxy"
Oct 29 23:35:23.741944 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.741932    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="a37b6f4c-bacc-4584-bbf2-b2414fa85897" containerName="main"
Oct 29 23:35:23.742041 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.742030    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="f45f800d-3877-415d-b832-92956ed12f88" containerName="istio-proxy"
Oct 29 23:35:23.742129 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.742118    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" containerName="istio-proxy"
Oct 29 23:35:23.742273 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.742259    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcfb74ac-619a-419e-8284-25b3f1a061b9" containerName="istio-proxy"
Oct 29 23:35:23.743087 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.743068    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a8fbb40-37c6-44ef-8339-5bd0187e687b" containerName="revoke-keys"
Oct 29 23:35:23.743237 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.743224    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="49d459ae-07ae-4c53-aae6-934c25119c6c" containerName="istio-proxy"
Oct 29 23:35:23.743336 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.743325    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5e71c48-e2a9-4c91-bf79-b5c5baef9c89" containerName="istio-proxy"
Oct 29 23:35:23.743437 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.743425    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3b29afd-299b-4dfc-b78f-73981c1c343a" containerName="main"
Oct 29 23:35:23.743521 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.743512    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="df0c1ec4-67dd-424e-97bf-91e504744d37" containerName="maas-api"
Oct 29 23:35:23.743595 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.743586    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="680505f8-f92f-4225-b61f-5b323c23b236" containerName="main"
Oct 29 23:35:23.778896 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.764143    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:23.859762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.857359    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"]
Oct 29 23:35:23.867764 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.867065    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:35:23.867974 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.867955    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:35:23.868149 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.868133    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:35:23.868382 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.868366    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:35:23.871453 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.869839    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-60cead78-h9ktd\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:23.871453 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.870041    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkd85\" (UniqueName: \"kubernetes.io/projected/ab380e45-5b8c-437f-a387-80d5194066d2-kube-api-access-wkd85\") pod \"maas-api-revoke-sub-60cead78-h9ktd\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:23.871453 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.870127    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-60cead78-h9ktd\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:23.971648 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.971591    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-60cead78-h9ktd\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:23.971898 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.971804    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkd85\" (UniqueName: \"kubernetes.io/projected/ab380e45-5b8c-437f-a387-80d5194066d2-kube-api-access-wkd85\") pod \"maas-api-revoke-sub-60cead78-h9ktd\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:23.971898 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.971875    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-60cead78-h9ktd\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:23.972978 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.972955    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-60cead78-h9ktd\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:23.980460 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.980407    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-60cead78-h9ktd\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:23.995798 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:23.995772    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkd85\" (UniqueName: \"kubernetes.io/projected/ab380e45-5b8c-437f-a387-80d5194066d2-kube-api-access-wkd85\") pod \"maas-api-revoke-sub-60cead78-h9ktd\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:24.184265 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:24.184227    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:24.459907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:24.443318    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49d459ae-07ae-4c53-aae6-934c25119c6c" path="/var/lib/kubelet/pods/49d459ae-07ae-4c53-aae6-934c25119c6c/volumes"
Oct 29 23:35:24.459907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:24.444090    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c3d6233-ff6f-4821-861f-9dd0eacbbe6b" path="/var/lib/kubelet/pods/7c3d6233-ff6f-4821-861f-9dd0eacbbe6b/volumes"
Oct 29 23:35:24.459907 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:24.444801    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f45f800d-3877-415d-b832-92956ed12f88" path="/var/lib/kubelet/pods/f45f800d-3877-415d-b832-92956ed12f88/volumes"
Oct 29 23:35:24.642489 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:24.621655    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"]
Oct 29 23:35:25.088043 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:25.088007    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd" event={"ID":"ab380e45-5b8c-437f-a387-80d5194066d2","Type":"ContainerStarted","Data":"2686c919f8c193d9413175e91fab970c06427a34e91bc86bc484f7c993dd4ad2"}
Oct 29 23:35:25.088256 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:25.088068    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd" event={"ID":"ab380e45-5b8c-437f-a387-80d5194066d2","Type":"ContainerStarted","Data":"c56842a7c0163e47f40fcff5ee4072f075df7191055b93abd4c4e630cbfe1542"}
Oct 29 23:35:25.131847 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:25.131796    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd" podStartSLOduration=2.131781908 podStartE2EDuration="2.131781908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:35:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:35:25.129995523 +0000 UTC m=+1304.825104403" watchObservedRunningTime="2026-10-29 23:35:25.131781908 +0000 UTC m=+1304.826890784"
Oct 29 23:35:26.096337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:26.096303    6040 generic.go:356] "Generic (PLEG): container finished" podID="ab380e45-5b8c-437f-a387-80d5194066d2" containerID="2686c919f8c193d9413175e91fab970c06427a34e91bc86bc484f7c993dd4ad2" exitCode=0
Oct 29 23:35:26.096780 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:26.096390    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd" event={"ID":"ab380e45-5b8c-437f-a387-80d5194066d2","Type":"ContainerDied","Data":"2686c919f8c193d9413175e91fab970c06427a34e91bc86bc484f7c993dd4ad2"}
Oct 29 23:35:27.252754 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:27.252712    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:27.327722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:27.327690    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-xks-ca\") pod \"ab380e45-5b8c-437f-a387-80d5194066d2\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") "
Oct 29 23:35:27.327899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:27.327876    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-openshift-ca\") pod \"ab380e45-5b8c-437f-a387-80d5194066d2\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") "
Oct 29 23:35:27.327979 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:27.327911    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab380e45-5b8c-437f-a387-80d5194066d2-kube-api-access-wkd85\" (UniqueName: \"kubernetes.io/projected/ab380e45-5b8c-437f-a387-80d5194066d2-kube-api-access-wkd85\") pod \"ab380e45-5b8c-437f-a387-80d5194066d2\" (UID: \"ab380e45-5b8c-437f-a387-80d5194066d2\") "
Oct 29 23:35:27.328305 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:27.328269    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-openshift-ca" pod "ab380e45-5b8c-437f-a387-80d5194066d2" (UID: "ab380e45-5b8c-437f-a387-80d5194066d2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:27.330035 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:27.329996    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-xks-ca" pod "ab380e45-5b8c-437f-a387-80d5194066d2" (UID: "ab380e45-5b8c-437f-a387-80d5194066d2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:35:27.330131 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:27.330113    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab380e45-5b8c-437f-a387-80d5194066d2-kube-api-access-wkd85" pod "ab380e45-5b8c-437f-a387-80d5194066d2" (UID: "ab380e45-5b8c-437f-a387-80d5194066d2"). InnerVolumeSpecName "kube-api-access-wkd85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:27.429511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:27.429471    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:27.429511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:27.429505    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkd85\" (UniqueName: \"kubernetes.io/projected/ab380e45-5b8c-437f-a387-80d5194066d2-kube-api-access-wkd85\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:27.429709 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:27.429519    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab380e45-5b8c-437f-a387-80d5194066d2-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:28.110409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:28.110375    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"
Oct 29 23:35:28.110409 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:28.110402    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd" event={"ID":"ab380e45-5b8c-437f-a387-80d5194066d2","Type":"ContainerDied","Data":"c56842a7c0163e47f40fcff5ee4072f075df7191055b93abd4c4e630cbfe1542"}
Oct 29 23:35:28.110606 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:28.110447    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c56842a7c0163e47f40fcff5ee4072f075df7191055b93abd4c4e630cbfe1542"
Oct 29 23:35:28.840449 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:28.840386    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:35:28.946580 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:28.946551    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-kube-api-access-pwgdq\" (UniqueName: \"kubernetes.io/projected/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-kube-api-access-pwgdq\") pod \"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef\" (UID: \"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef\") "
Oct 29 23:35:28.946758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:28.946654    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-plugins-config-volume\") pod \"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef\" (UID: \"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef\") "
Oct 29 23:35:28.947044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:28.947019    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-plugins-config-volume" pod "7bbff9d3-81f5-4f3f-9be1-afba3a8058ef" (UID: "7bbff9d3-81f5-4f3f-9be1-afba3a8058ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:28.948525 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:28.948506    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-kube-api-access-pwgdq" pod "7bbff9d3-81f5-4f3f-9be1-afba3a8058ef" (UID: "7bbff9d3-81f5-4f3f-9be1-afba3a8058ef"). InnerVolumeSpecName "kube-api-access-pwgdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:29.048504 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:29.048470    6040 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-plugins-config-volume\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:29.048504 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:29.048498    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwgdq\" (UniqueName: \"kubernetes.io/projected/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef-kube-api-access-pwgdq\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:29.117899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:29.117811    6040 generic.go:356] "Generic (PLEG): container finished" podID="7bbff9d3-81f5-4f3f-9be1-afba3a8058ef" containerID="e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377" exitCode=0
Oct 29 23:35:29.117899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:29.117857    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh" event={"ID":"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef","Type":"ContainerDied","Data":"e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377"}
Oct 29 23:35:29.117899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:29.117869    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"
Oct 29 23:35:29.118139 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:29.117909    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh" event={"ID":"7bbff9d3-81f5-4f3f-9be1-afba3a8058ef","Type":"ContainerDied","Data":"f56b5001ec7ec7cff66e2c5ecb8c79857679b134767e97ac7e71a2c60ae2d8b3"}
Oct 29 23:35:29.118139 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:29.117938    6040 scope.go:122] "RemoveContainer" containerID="e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377"
Oct 29 23:35:29.128313 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:29.128295    6040 scope.go:122] "RemoveContainer" containerID="e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377"
Oct 29 23:35:29.128556 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:29.128538    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377\": container with ID starting with e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377 not found: ID does not exist" containerID="e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377"
Oct 29 23:35:29.128604 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:29.128565    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377"} err="failed to get container status \"e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377\": rpc error: code = NotFound desc = could not find container \"e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377\": container with ID starting with e2b6f7964f2a3635ef7484a32604c04a5e031d77304c811f9a4ee47ce39ec377 not found: ID does not exist"
Oct 29 23:35:29.172268 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:29.172236    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"]
Oct 29 23:35:29.183971 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:29.183944    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ae1784-7b68c568cf-rhvbh"]
Oct 29 23:35:29.780231 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:29.780197    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:35:30.427684 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:30.427650    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bbff9d3-81f5-4f3f-9be1-afba3a8058ef" path="/var/lib/kubelet/pods/7bbff9d3-81f5-4f3f-9be1-afba3a8058ef/volumes"
Oct 29 23:35:31.258486 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.258447    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t" podUID="186078cc-b997-4651-9dcf-b33603f13047" containerName="main" containerID="cri-o://6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c" gracePeriod=45
Oct 29 23:35:31.272176 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.272142    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" containerID="cri-o://02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23" gracePeriod=45
Oct 29 23:35:31.281671 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.281626    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" containerID="cri-o://1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e" gracePeriod=45
Oct 29 23:35:31.285702 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.285675    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" containerID="cri-o://cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4" gracePeriod=45
Oct 29 23:35:31.362045 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.362013    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": dial tcp 10.133.0.49:8000: connect: connection refused"
Oct 29 23:35:31.362393 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.362364    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 29 23:35:31.399541 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.399101    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": dial tcp 10.133.0.80:8000: connect: connection refused"
Oct 29 23:35:31.606335 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.606304    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:35:31.628816 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.628786    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:35:31.636145 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.636025    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:35:31.649844 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.649805    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:35:31.677416 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.677386    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-home\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-home\") pod \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") "
Oct 29 23:35:31.677589 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.677430    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc92da1f-4308-4050-8c1f-7a926c5eae0f-kube-api-access-7j5ln\" (UniqueName: \"kubernetes.io/projected/dc92da1f-4308-4050-8c1f-7a926c5eae0f-kube-api-access-7j5ln\") pod \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") "
Oct 29 23:35:31.677589 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.677472    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tls-certs\" (UniqueName: \"kubernetes.io/secret/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tls-certs\") pod \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") "
Oct 29 23:35:31.677589 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.677499    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-home\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-home\") pod \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") "
Oct 29 23:35:31.677589 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.677516    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-model-cache\") pod \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") "
Oct 29 23:35:31.677836 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.677719    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-home" pod "a0d3ee3e-07bd-4868-a39a-dccab1fc6222" (UID: "a0d3ee3e-07bd-4868-a39a-dccab1fc6222"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.677971 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.677946    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-dshm\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-dshm\") pod \"186078cc-b997-4651-9dcf-b33603f13047\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") "
Oct 29 23:35:31.678060 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678009    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-model-cache" pod "dc92da1f-4308-4050-8c1f-7a926c5eae0f" (UID: "dc92da1f-4308-4050-8c1f-7a926c5eae0f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.678060 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678014    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-model-cache\") pod \"186078cc-b997-4651-9dcf-b33603f13047\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") "
Oct 29 23:35:31.678060 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678030    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-home" pod "dc92da1f-4308-4050-8c1f-7a926c5eae0f" (UID: "dc92da1f-4308-4050-8c1f-7a926c5eae0f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.678235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678073    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-dshm\") pod \"a292574a-72a5-427c-a344-255fac8aa567\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") "
Oct 29 23:35:31.678235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678106    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/186078cc-b997-4651-9dcf-b33603f13047-kube-api-access-w5qwz\" (UniqueName: \"kubernetes.io/projected/186078cc-b997-4651-9dcf-b33603f13047-kube-api-access-w5qwz\") pod \"186078cc-b997-4651-9dcf-b33603f13047\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") "
Oct 29 23:35:31.678235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678136    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-dshm\") pod \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") "
Oct 29 23:35:31.678235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678186    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-home\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-home\") pod \"a292574a-72a5-427c-a344-255fac8aa567\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") "
Oct 29 23:35:31.678235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678202    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-model-cache" pod "186078cc-b997-4651-9dcf-b33603f13047" (UID: "186078cc-b997-4651-9dcf-b33603f13047"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.678235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678217    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/186078cc-b997-4651-9dcf-b33603f13047-tls-certs\" (UniqueName: \"kubernetes.io/secret/186078cc-b997-4651-9dcf-b33603f13047-tls-certs\") pod \"186078cc-b997-4651-9dcf-b33603f13047\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") "
Oct 29 23:35:31.678547 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678251    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-home\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-home\") pod \"186078cc-b997-4651-9dcf-b33603f13047\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") "
Oct 29 23:35:31.678547 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678317    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a292574a-72a5-427c-a344-255fac8aa567-tls-certs\" (UniqueName: \"kubernetes.io/secret/a292574a-72a5-427c-a344-255fac8aa567-tls-certs\") pod \"a292574a-72a5-427c-a344-255fac8aa567\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") "
Oct 29 23:35:31.678547 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678437    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a292574a-72a5-427c-a344-255fac8aa567-kube-api-access-nmr2q\" (UniqueName: \"kubernetes.io/projected/a292574a-72a5-427c-a344-255fac8aa567-kube-api-access-nmr2q\") pod \"a292574a-72a5-427c-a344-255fac8aa567\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") "
Oct 29 23:35:31.678547 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678475    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-tmp-dir\") pod \"186078cc-b997-4651-9dcf-b33603f13047\" (UID: \"186078cc-b997-4651-9dcf-b33603f13047\") "
Oct 29 23:35:31.678547 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678509    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-kube-api-access-4d4z6\" (UniqueName: \"kubernetes.io/projected/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-kube-api-access-4d4z6\") pod \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") "
Oct 29 23:35:31.678849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678564    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-tmp-dir\") pod \"a292574a-72a5-427c-a344-255fac8aa567\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") "
Oct 29 23:35:31.678849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678621    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tls-certs\") pod \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") "
Oct 29 23:35:31.678849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678657    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tmp-dir\") pod \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") "
Oct 29 23:35:31.678849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678693    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tmp-dir\") pod \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") "
Oct 29 23:35:31.678849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678762    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-model-cache\") pod \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\" (UID: \"a0d3ee3e-07bd-4868-a39a-dccab1fc6222\") "
Oct 29 23:35:31.678849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678792    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-model-cache\") pod \"a292574a-72a5-427c-a344-255fac8aa567\" (UID: \"a292574a-72a5-427c-a344-255fac8aa567\") "
Oct 29 23:35:31.678849 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.678828    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-dshm\") pod \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\" (UID: \"dc92da1f-4308-4050-8c1f-7a926c5eae0f\") "
Oct 29 23:35:31.679579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.679360    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.679579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.679389    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.679579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.679402    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.679579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.679416    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.679866 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.679800    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-tmp-dir" pod "186078cc-b997-4651-9dcf-b33603f13047" (UID: "186078cc-b997-4651-9dcf-b33603f13047"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.680239 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.680130    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tmp-dir" pod "a0d3ee3e-07bd-4868-a39a-dccab1fc6222" (UID: "a0d3ee3e-07bd-4868-a39a-dccab1fc6222"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.680586 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.680425    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-home" pod "186078cc-b997-4651-9dcf-b33603f13047" (UID: "186078cc-b997-4651-9dcf-b33603f13047"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.680586 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.680498    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-tmp-dir" pod "a292574a-72a5-427c-a344-255fac8aa567" (UID: "a292574a-72a5-427c-a344-255fac8aa567"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.680586 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.680541    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tmp-dir" pod "dc92da1f-4308-4050-8c1f-7a926c5eae0f" (UID: "dc92da1f-4308-4050-8c1f-7a926c5eae0f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.680800 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.680608    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tls-certs" pod "dc92da1f-4308-4050-8c1f-7a926c5eae0f" (UID: "dc92da1f-4308-4050-8c1f-7a926c5eae0f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:35:31.680862 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.680831    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-model-cache" pod "a0d3ee3e-07bd-4868-a39a-dccab1fc6222" (UID: "a0d3ee3e-07bd-4868-a39a-dccab1fc6222"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.681211 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.681182    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-model-cache" pod "a292574a-72a5-427c-a344-255fac8aa567" (UID: "a292574a-72a5-427c-a344-255fac8aa567"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.681293 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.681236    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-home" pod "a292574a-72a5-427c-a344-255fac8aa567" (UID: "a292574a-72a5-427c-a344-255fac8aa567"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.681898 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.681874    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-dshm" pod "186078cc-b997-4651-9dcf-b33603f13047" (UID: "186078cc-b997-4651-9dcf-b33603f13047"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.683858 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.683832    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tls-certs" pod "a0d3ee3e-07bd-4868-a39a-dccab1fc6222" (UID: "a0d3ee3e-07bd-4868-a39a-dccab1fc6222"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:35:31.684672 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.684631    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a292574a-72a5-427c-a344-255fac8aa567-kube-api-access-nmr2q" pod "a292574a-72a5-427c-a344-255fac8aa567" (UID: "a292574a-72a5-427c-a344-255fac8aa567"). InnerVolumeSpecName "kube-api-access-nmr2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:31.685245 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.685195    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186078cc-b997-4651-9dcf-b33603f13047-tls-certs" pod "186078cc-b997-4651-9dcf-b33603f13047" (UID: "186078cc-b997-4651-9dcf-b33603f13047"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:35:31.685245 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.685196    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/186078cc-b997-4651-9dcf-b33603f13047-kube-api-access-w5qwz" pod "186078cc-b997-4651-9dcf-b33603f13047" (UID: "186078cc-b997-4651-9dcf-b33603f13047"). InnerVolumeSpecName "kube-api-access-w5qwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:31.685398 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.685260    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a292574a-72a5-427c-a344-255fac8aa567-tls-certs" pod "a292574a-72a5-427c-a344-255fac8aa567" (UID: "a292574a-72a5-427c-a344-255fac8aa567"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:35:31.685398 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.685355    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc92da1f-4308-4050-8c1f-7a926c5eae0f-kube-api-access-7j5ln" pod "dc92da1f-4308-4050-8c1f-7a926c5eae0f" (UID: "dc92da1f-4308-4050-8c1f-7a926c5eae0f"). InnerVolumeSpecName "kube-api-access-7j5ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:31.685398 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.685387    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-dshm" pod "a0d3ee3e-07bd-4868-a39a-dccab1fc6222" (UID: "a0d3ee3e-07bd-4868-a39a-dccab1fc6222"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.685788 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.685409    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-dshm" pod "a292574a-72a5-427c-a344-255fac8aa567" (UID: "a292574a-72a5-427c-a344-255fac8aa567"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.685788 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.685507    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-kube-api-access-4d4z6" pod "a0d3ee3e-07bd-4868-a39a-dccab1fc6222" (UID: "a0d3ee3e-07bd-4868-a39a-dccab1fc6222"). InnerVolumeSpecName "kube-api-access-4d4z6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:31.687163 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.687140    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-dshm" pod "dc92da1f-4308-4050-8c1f-7a926c5eae0f" (UID: "dc92da1f-4308-4050-8c1f-7a926c5eae0f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787386    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a292574a-72a5-427c-a344-255fac8aa567-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787428    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmr2q\" (UniqueName: \"kubernetes.io/projected/a292574a-72a5-427c-a344-255fac8aa567-kube-api-access-nmr2q\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787444    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787458    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d4z6\" (UniqueName: \"kubernetes.io/projected/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-kube-api-access-4d4z6\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787474    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787493    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787506    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787518    6040 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-tmp-dir\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787530    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787547    6040 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-model-cache\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787558    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc92da1f-4308-4050-8c1f-7a926c5eae0f-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787571    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j5ln\" (UniqueName: \"kubernetes.io/projected/dc92da1f-4308-4050-8c1f-7a926c5eae0f-kube-api-access-7j5ln\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787583    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc92da1f-4308-4050-8c1f-7a926c5eae0f-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787599    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787611    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787623    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5qwz\" (UniqueName: \"kubernetes.io/projected/186078cc-b997-4651-9dcf-b33603f13047-kube-api-access-w5qwz\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787636    6040 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0d3ee3e-07bd-4868-a39a-dccab1fc6222-dshm\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787653    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a292574a-72a5-427c-a344-255fac8aa567-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787665    6040 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/186078cc-b997-4651-9dcf-b33603f13047-tls-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:31.795807 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:31.787676    6040 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/186078cc-b997-4651-9dcf-b33603f13047-home\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:32.140984 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.140945    6040 generic.go:356] "Generic (PLEG): container finished" podID="a292574a-72a5-427c-a344-255fac8aa567" containerID="02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23" exitCode=0
Oct 29 23:35:32.141134 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.141007    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"
Oct 29 23:35:32.141134 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.141028    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" event={"ID":"a292574a-72a5-427c-a344-255fac8aa567","Type":"ContainerDied","Data":"02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23"}
Oct 29 23:35:32.141134 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.141074    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg" event={"ID":"a292574a-72a5-427c-a344-255fac8aa567","Type":"ContainerDied","Data":"18e0190dd42548ebf16b88aab524a1f414c2540664bb7671c8cca1f6dc4539d7"}
Oct 29 23:35:32.141134 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.141096    6040 scope.go:122] "RemoveContainer" containerID="02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23"
Oct 29 23:35:32.142488 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.142467    6040 generic.go:356] "Generic (PLEG): container finished" podID="186078cc-b997-4651-9dcf-b33603f13047" containerID="6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c" exitCode=0
Oct 29 23:35:32.142603 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.142532    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"
Oct 29 23:35:32.142603 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.142549    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t" event={"ID":"186078cc-b997-4651-9dcf-b33603f13047","Type":"ContainerDied","Data":"6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c"}
Oct 29 23:35:32.142603 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.142589    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t" event={"ID":"186078cc-b997-4651-9dcf-b33603f13047","Type":"ContainerDied","Data":"7e24f2fb28429bccea377b3e2a12650385c5df872b6547f8121fe4a3d4479371"}
Oct 29 23:35:32.144133 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.144109    6040 generic.go:356] "Generic (PLEG): container finished" podID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerID="1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e" exitCode=0
Oct 29 23:35:32.144257 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.144162    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"
Oct 29 23:35:32.144257 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.144162    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" event={"ID":"dc92da1f-4308-4050-8c1f-7a926c5eae0f","Type":"ContainerDied","Data":"1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e"}
Oct 29 23:35:32.144359 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.144260    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp" event={"ID":"dc92da1f-4308-4050-8c1f-7a926c5eae0f","Type":"ContainerDied","Data":"5c94468d30e385b9c0888682b58814ab56d3e5c0f7a4441eb0d4217e6d88210a"}
Oct 29 23:35:32.145957 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.145931    6040 generic.go:356] "Generic (PLEG): container finished" podID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerID="cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4" exitCode=0
Oct 29 23:35:32.146065 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.145960    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" event={"ID":"a0d3ee3e-07bd-4868-a39a-dccab1fc6222","Type":"ContainerDied","Data":"cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4"}
Oct 29 23:35:32.146065 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.145983    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"
Oct 29 23:35:32.146065 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.145997    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv" event={"ID":"a0d3ee3e-07bd-4868-a39a-dccab1fc6222","Type":"ContainerDied","Data":"1655f6e66501a3d4190592eb844069e9b3107535c6029bbcca04eebbe2d9293f"}
Oct 29 23:35:32.153295 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.153276    6040 scope.go:122] "RemoveContainer" containerID="02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23"
Oct 29 23:35:32.153562 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:32.153537    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23\": container with ID starting with 02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23 not found: ID does not exist" containerID="02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23"
Oct 29 23:35:32.153621 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.153572    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23"} err="failed to get container status \"02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23\": rpc error: code = NotFound desc = could not find container \"02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23\": container with ID starting with 02dba5393d75ab65524051cdd46adccc6f37afc284555cfa3f050240c91a7d23 not found: ID does not exist"
Oct 29 23:35:32.153621 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.153594    6040 scope.go:122] "RemoveContainer" containerID="6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c"
Oct 29 23:35:32.163676 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.163660    6040 scope.go:122] "RemoveContainer" containerID="6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c"
Oct 29 23:35:32.163925 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:32.163903    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c\": container with ID starting with 6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c not found: ID does not exist" containerID="6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c"
Oct 29 23:35:32.164017 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.163927    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c"} err="failed to get container status \"6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c\": rpc error: code = NotFound desc = could not find container \"6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c\": container with ID starting with 6a0add10b46457a56cfaf698bf21fa35125b2aca3599e1cc2c867f9c3b297f5c not found: ID does not exist"
Oct 29 23:35:32.164017 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.163942    6040 scope.go:122] "RemoveContainer" containerID="1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e"
Oct 29 23:35:32.172437 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.172422    6040 scope.go:122] "RemoveContainer" containerID="1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e"
Oct 29 23:35:32.172698 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:32.172681    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e\": container with ID starting with 1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e not found: ID does not exist" containerID="1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e"
Oct 29 23:35:32.172758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.172704    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e"} err="failed to get container status \"1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e\": rpc error: code = NotFound desc = could not find container \"1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e\": container with ID starting with 1815b04a2f4ba8c57fecafb4c628ea55ba9912b1a8b9f71fbad3dcc77ae8125e not found: ID does not exist"
Oct 29 23:35:32.172758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.172720    6040 scope.go:122] "RemoveContainer" containerID="cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4"
Oct 29 23:35:32.180683 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.180657    6040 scope.go:122] "RemoveContainer" containerID="cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4"
Oct 29 23:35:32.180975 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:32.180951    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4\": container with ID starting with cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4 not found: ID does not exist" containerID="cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4"
Oct 29 23:35:32.181073 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.180983    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4"} err="failed to get container status \"cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4\": rpc error: code = NotFound desc = could not find container \"cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4\": container with ID starting with cea24a40a9e3a8a2fb2139d0210bb4037afec2371df073fbd36bfee4571003b4 not found: ID does not exist"
Oct 29 23:35:32.223664 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.223631    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"]
Oct 29 23:35:32.236065 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.236025    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/premium-w4-aa7e96-kserve-6985bb7df7-jbxjg"]
Oct 29 23:35:32.289348 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.289278    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"]
Oct 29 23:35:32.289539 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.289515    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy" containerID="cri-o://fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23" gracePeriod=30
Oct 29 23:35:32.296374 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.296342    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"]
Oct 29 23:35:32.308899 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.308874    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-w4-aa7e96-kserve-76df4bff5b-8bfvp"]
Oct 29 23:35:32.311763 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.311723    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"]
Oct 29 23:35:32.311971 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.311950    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy" containerID="cri-o://731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4" gracePeriod=30
Oct 29 23:35:32.319494 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.319467    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"]
Oct 29 23:35:32.319688 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.319666    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerName="istio-proxy" containerID="cri-o://beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e" gracePeriod=30
Oct 29 23:35:32.341127 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.341093    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"]
Oct 29 23:35:32.347585 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.347558    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/simulator-w4-aa7e96-kserve-69c84f6756-75hxv"]
Oct 29 23:35:32.382808 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.381343    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"]
Oct 29 23:35:32.384966 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.384936    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t"]
Oct 29 23:35:32.444016 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.443924    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="186078cc-b997-4651-9dcf-b33603f13047" path="/var/lib/kubelet/pods/186078cc-b997-4651-9dcf-b33603f13047/volumes"
Oct 29 23:35:32.444548 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.444527    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" path="/var/lib/kubelet/pods/a0d3ee3e-07bd-4868-a39a-dccab1fc6222/volumes"
Oct 29 23:35:32.445120 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.445088    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a292574a-72a5-427c-a344-255fac8aa567" path="/var/lib/kubelet/pods/a292574a-72a5-427c-a344-255fac8aa567/volumes"
Oct 29 23:35:32.446123 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.446098    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" path="/var/lib/kubelet/pods/dc92da1f-4308-4050-8c1f-7a926c5eae0f/volumes"
Oct 29 23:35:32.463078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.463050    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 29 23:35:32.463186 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.463105    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:35:32.520049 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:32.520009    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-aa7e96/distinct-2-w4-aa7e96-kserve-548b69cd75-qn67t" podUID="186078cc-b997-4651-9dcf-b33603f13047" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:35:34.979274 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:34.979210    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 29 23:35:34.979928 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:34.979308    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:35:34.999405 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:34.999362    6040 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 29 23:35:34.999602 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:34.999437    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:35:37.595580 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.595543    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:35:37.645273 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.645247    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:35:37.648144 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.648118    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:35:37.648521 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.648499    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-credential-socket\") pod \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") "
Oct 29 23:35:37.648622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.648572    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-socket\") pod \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") "
Oct 29 23:35:37.648683 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.648631    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-podinfo\") pod \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") "
Oct 29 23:35:37.648720 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.648683    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-data\") pod \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") "
Oct 29 23:35:37.648813 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.648784    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-credential-socket" pod "4d9dcc91-f5e8-47fb-801e-251be89b55d5" (UID: "4d9dcc91-f5e8-47fb-801e-251be89b55d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.648813 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.648801    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-kube-api-access-gbdq9\" (UniqueName: \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-kube-api-access-gbdq9\") pod \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") "
Oct 29 23:35:37.648926 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.648837    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-certs\") pod \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") "
Oct 29 23:35:37.648926 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.648880    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-token\" (UniqueName: \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-token\") pod \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") "
Oct 29 23:35:37.649024 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.648952    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-envoy\") pod \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") "
Oct 29 23:35:37.649024 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.648993    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istiod-ca-cert\") pod \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\" (UID: \"4d9dcc91-f5e8-47fb-801e-251be89b55d5\") "
Oct 29 23:35:37.649299 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.649273    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-socket" pod "4d9dcc91-f5e8-47fb-801e-251be89b55d5" (UID: "4d9dcc91-f5e8-47fb-801e-251be89b55d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.649391 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.649355    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-certs" pod "4d9dcc91-f5e8-47fb-801e-251be89b55d5" (UID: "4d9dcc91-f5e8-47fb-801e-251be89b55d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.649391 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.649373    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-data" pod "4d9dcc91-f5e8-47fb-801e-251be89b55d5" (UID: "4d9dcc91-f5e8-47fb-801e-251be89b55d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.649515 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.649413    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.649515 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.649434    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.649617 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.649596    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istiod-ca-cert" pod "4d9dcc91-f5e8-47fb-801e-251be89b55d5" (UID: "4d9dcc91-f5e8-47fb-801e-251be89b55d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:37.651469 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.651445    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-envoy" pod "4d9dcc91-f5e8-47fb-801e-251be89b55d5" (UID: "4d9dcc91-f5e8-47fb-801e-251be89b55d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.651958 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.651927    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-podinfo" pod "4d9dcc91-f5e8-47fb-801e-251be89b55d5" (UID: "4d9dcc91-f5e8-47fb-801e-251be89b55d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:37.652048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.651967    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-token" pod "4d9dcc91-f5e8-47fb-801e-251be89b55d5" (UID: "4d9dcc91-f5e8-47fb-801e-251be89b55d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:37.652109 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.652012    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-kube-api-access-gbdq9" pod "4d9dcc91-f5e8-47fb-801e-251be89b55d5" (UID: "4d9dcc91-f5e8-47fb-801e-251be89b55d5"). InnerVolumeSpecName "kube-api-access-gbdq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:37.750323 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750285    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-socket\") pod \"256870d0-9dce-4e09-8374-9835d54c4f78\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750338    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-token\" (UniqueName: \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-token\") pod \"cc61fe28-ed42-4655-99de-d63d17a30a3f\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750379    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-envoy\") pod \"cc61fe28-ed42-4655-99de-d63d17a30a3f\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750416    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-credential-socket\") pod \"cc61fe28-ed42-4655-99de-d63d17a30a3f\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750475    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-istio-token\" (UniqueName: \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-istio-token\") pod \"256870d0-9dce-4e09-8374-9835d54c4f78\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750526    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-kube-api-access-znt72\" (UniqueName: \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-kube-api-access-znt72\") pod \"cc61fe28-ed42-4655-99de-d63d17a30a3f\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750552    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/256870d0-9dce-4e09-8374-9835d54c4f78-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/256870d0-9dce-4e09-8374-9835d54c4f78-istio-podinfo\") pod \"256870d0-9dce-4e09-8374-9835d54c4f78\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750584    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-data\") pod \"256870d0-9dce-4e09-8374-9835d54c4f78\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750593    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-socket" pod "256870d0-9dce-4e09-8374-9835d54c4f78" (UID: "256870d0-9dce-4e09-8374-9835d54c4f78"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750647    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/256870d0-9dce-4e09-8374-9835d54c4f78-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/256870d0-9dce-4e09-8374-9835d54c4f78-istiod-ca-cert\") pod \"256870d0-9dce-4e09-8374-9835d54c4f78\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750695    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-podinfo\") pod \"cc61fe28-ed42-4655-99de-d63d17a30a3f\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750772    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-certs\") pod \"256870d0-9dce-4e09-8374-9835d54c4f78\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750809    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-envoy\") pod \"256870d0-9dce-4e09-8374-9835d54c4f78\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750843    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-data\") pod \"cc61fe28-ed42-4655-99de-d63d17a30a3f\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750874    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-credential-socket\") pod \"256870d0-9dce-4e09-8374-9835d54c4f78\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750917    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-kube-api-access-7rhr5\" (UniqueName: \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-kube-api-access-7rhr5\") pod \"256870d0-9dce-4e09-8374-9835d54c4f78\" (UID: \"256870d0-9dce-4e09-8374-9835d54c4f78\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.750951    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-certs\") pod \"cc61fe28-ed42-4655-99de-d63d17a30a3f\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.751009    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-socket\") pod \"cc61fe28-ed42-4655-99de-d63d17a30a3f\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") "
Oct 29 23:35:37.751075 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.751044    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc61fe28-ed42-4655-99de-d63d17a30a3f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc61fe28-ed42-4655-99de-d63d17a30a3f-istiod-ca-cert\") pod \"cc61fe28-ed42-4655-99de-d63d17a30a3f\" (UID: \"cc61fe28-ed42-4655-99de-d63d17a30a3f\") "
Oct 29 23:35:37.752758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.751490    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.752758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.751515    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.752758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.751529    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbdq9\" (UniqueName: \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-kube-api-access-gbdq9\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.752758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.751545    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.752758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.751562    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.752758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.751575    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.752758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.751592    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.752758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.751606    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d9dcc91-f5e8-47fb-801e-251be89b55d5-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.752758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.752692    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/256870d0-9dce-4e09-8374-9835d54c4f78-istiod-ca-cert" pod "256870d0-9dce-4e09-8374-9835d54c4f78" (UID: "256870d0-9dce-4e09-8374-9835d54c4f78"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:37.753216 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.753025    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc61fe28-ed42-4655-99de-d63d17a30a3f-istiod-ca-cert" pod "cc61fe28-ed42-4655-99de-d63d17a30a3f" (UID: "cc61fe28-ed42-4655-99de-d63d17a30a3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:37.753216 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.753202    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-credential-socket" pod "256870d0-9dce-4e09-8374-9835d54c4f78" (UID: "256870d0-9dce-4e09-8374-9835d54c4f78"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.753327 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.753290    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-token" pod "cc61fe28-ed42-4655-99de-d63d17a30a3f" (UID: "cc61fe28-ed42-4655-99de-d63d17a30a3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:37.753391 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.753332    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-data" pod "256870d0-9dce-4e09-8374-9835d54c4f78" (UID: "256870d0-9dce-4e09-8374-9835d54c4f78"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.753990 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.753487    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-certs" pod "256870d0-9dce-4e09-8374-9835d54c4f78" (UID: "256870d0-9dce-4e09-8374-9835d54c4f78"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.753990 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.753958    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-data" pod "cc61fe28-ed42-4655-99de-d63d17a30a3f" (UID: "cc61fe28-ed42-4655-99de-d63d17a30a3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.754199 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.754034    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-istio-token" pod "256870d0-9dce-4e09-8374-9835d54c4f78" (UID: "256870d0-9dce-4e09-8374-9835d54c4f78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:37.754199 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.754054    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-socket" pod "cc61fe28-ed42-4655-99de-d63d17a30a3f" (UID: "cc61fe28-ed42-4655-99de-d63d17a30a3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.754199 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.754091    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-credential-socket" pod "cc61fe28-ed42-4655-99de-d63d17a30a3f" (UID: "cc61fe28-ed42-4655-99de-d63d17a30a3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.754199 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.754160    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-certs" pod "cc61fe28-ed42-4655-99de-d63d17a30a3f" (UID: "cc61fe28-ed42-4655-99de-d63d17a30a3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.754515 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.754484    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-kube-api-access-7rhr5" pod "256870d0-9dce-4e09-8374-9835d54c4f78" (UID: "256870d0-9dce-4e09-8374-9835d54c4f78"). InnerVolumeSpecName "kube-api-access-7rhr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:37.754974 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.754954    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-envoy" pod "cc61fe28-ed42-4655-99de-d63d17a30a3f" (UID: "cc61fe28-ed42-4655-99de-d63d17a30a3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.755290 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.755265    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/256870d0-9dce-4e09-8374-9835d54c4f78-istio-podinfo" pod "256870d0-9dce-4e09-8374-9835d54c4f78" (UID: "256870d0-9dce-4e09-8374-9835d54c4f78"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:37.756317 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.756297    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-kube-api-access-znt72" pod "cc61fe28-ed42-4655-99de-d63d17a30a3f" (UID: "cc61fe28-ed42-4655-99de-d63d17a30a3f"). InnerVolumeSpecName "kube-api-access-znt72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:37.756317 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.756301    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-podinfo" pod "cc61fe28-ed42-4655-99de-d63d17a30a3f" (UID: "cc61fe28-ed42-4655-99de-d63d17a30a3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:37.756514 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.756498    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-envoy" pod "256870d0-9dce-4e09-8374-9835d54c4f78" (UID: "256870d0-9dce-4e09-8374-9835d54c4f78"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:37.853119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853042    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853074    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znt72\" (UniqueName: \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-kube-api-access-znt72\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853084    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/256870d0-9dce-4e09-8374-9835d54c4f78-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853093    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853101    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/256870d0-9dce-4e09-8374-9835d54c4f78-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853109    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853117    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853119 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853126    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853136    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853148    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/256870d0-9dce-4e09-8374-9835d54c4f78-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853156    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rhr5\" (UniqueName: \"kubernetes.io/projected/256870d0-9dce-4e09-8374-9835d54c4f78-kube-api-access-7rhr5\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853164    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853173    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853181    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc61fe28-ed42-4655-99de-d63d17a30a3f-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853189    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853197    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:37.853459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:37.853206    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc61fe28-ed42-4655-99de-d63d17a30a3f-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:38.206321 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.206286    6040 generic.go:356] "Generic (PLEG): container finished" podID="256870d0-9dce-4e09-8374-9835d54c4f78" containerID="731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4" exitCode=0
Oct 29 23:35:38.206500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.206330    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"
Oct 29 23:35:38.206500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.206375    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" event={"ID":"256870d0-9dce-4e09-8374-9835d54c4f78","Type":"ContainerDied","Data":"731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4"}
Oct 29 23:35:38.206500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.206473    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx" event={"ID":"256870d0-9dce-4e09-8374-9835d54c4f78","Type":"ContainerDied","Data":"aed31b23740ff0027d43c0c8f21375b7d0a5d8dfd2714e30e1616a69b6bf5fb5"}
Oct 29 23:35:38.206668 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.206503    6040 scope.go:122] "RemoveContainer" containerID="731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4"
Oct 29 23:35:38.207928 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.207904    6040 generic.go:356] "Generic (PLEG): container finished" podID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerID="fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23" exitCode=0
Oct 29 23:35:38.208154 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.207947    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"
Oct 29 23:35:38.208154 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.207943    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" event={"ID":"4d9dcc91-f5e8-47fb-801e-251be89b55d5","Type":"ContainerDied","Data":"fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23"}
Oct 29 23:35:38.208154 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.208065    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9" event={"ID":"4d9dcc91-f5e8-47fb-801e-251be89b55d5","Type":"ContainerDied","Data":"9125d86d73ae9ef9da29304915bb1e1b918676cb593a67b449f1285c64f4ba6b"}
Oct 29 23:35:38.209537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.209514    6040 generic.go:356] "Generic (PLEG): container finished" podID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerID="beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e" exitCode=0
Oct 29 23:35:38.209622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.209548    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"
Oct 29 23:35:38.209622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.209589    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" event={"ID":"cc61fe28-ed42-4655-99de-d63d17a30a3f","Type":"ContainerDied","Data":"beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e"}
Oct 29 23:35:38.209730 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.209634    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4" event={"ID":"cc61fe28-ed42-4655-99de-d63d17a30a3f","Type":"ContainerDied","Data":"02bf775635b2dbba634f53a3a3fe2b037f8dac3610b0b54d29803cd6606fa705"}
Oct 29 23:35:38.218659 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.218634    6040 scope.go:122] "RemoveContainer" containerID="731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4"
Oct 29 23:35:38.218977 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:38.218959    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4\": container with ID starting with 731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4 not found: ID does not exist" containerID="731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4"
Oct 29 23:35:38.219048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.218984    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4"} err="failed to get container status \"731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4\": rpc error: code = NotFound desc = could not find container \"731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4\": container with ID starting with 731f0c8897d44f842fd5f35cc8afbb230b511acdd6a9f9ce620cb76cdfb27ac4 not found: ID does not exist"
Oct 29 23:35:38.219048 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.219004    6040 scope.go:122] "RemoveContainer" containerID="fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23"
Oct 29 23:35:38.227852 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.227834    6040 scope.go:122] "RemoveContainer" containerID="fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23"
Oct 29 23:35:38.228105 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:38.228088    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23\": container with ID starting with fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23 not found: ID does not exist" containerID="fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23"
Oct 29 23:35:38.228172 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.228114    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23"} err="failed to get container status \"fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23\": rpc error: code = NotFound desc = could not find container \"fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23\": container with ID starting with fc7ca5c6535b6b4a1c3b3cc2f8ca194079d85e61d4b60629d2051a2018e81d23 not found: ID does not exist"
Oct 29 23:35:38.228172 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.228134    6040 scope.go:122] "RemoveContainer" containerID="beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e"
Oct 29 23:35:38.240013 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.239989    6040 scope.go:122] "RemoveContainer" containerID="beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e"
Oct 29 23:35:38.240265 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:38.240243    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e\": container with ID starting with beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e not found: ID does not exist" containerID="beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e"
Oct 29 23:35:38.240313 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.240271    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e"} err="failed to get container status \"beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e\": rpc error: code = NotFound desc = could not find container \"beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e\": container with ID starting with beb36bb25cb6d5c79cae85dd8a91348b85552d3cc3daa5edca6cc8ac81b9f00e not found: ID does not exist"
Oct 29 23:35:38.259918 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.259895    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"]
Oct 29 23:35:38.271783 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.271762    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-skhc4"]
Oct 29 23:35:38.301877 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.301851    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"]
Oct 29 23:35:38.305980 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.305960    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-5nbm9"]
Oct 29 23:35:38.339337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.339308    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"]
Oct 29 23:35:38.347542 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.347520    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-l49jx"]
Oct 29 23:35:38.424285 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.424252    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" path="/var/lib/kubelet/pods/256870d0-9dce-4e09-8374-9835d54c4f78/volumes"
Oct 29 23:35:38.424722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.424709    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" path="/var/lib/kubelet/pods/4d9dcc91-f5e8-47fb-801e-251be89b55d5/volumes"
Oct 29 23:35:38.425188 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.425173    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" path="/var/lib/kubelet/pods/cc61fe28-ed42-4655-99de-d63d17a30a3f/volumes"
Oct 29 23:35:38.589318 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:38.589233    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:35:44.450833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:44.450801    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"]
Oct 29 23:35:44.451320 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:44.451070    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy" containerID="cri-o://e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64" gracePeriod=30
Oct 29 23:35:44.458062 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:44.458033    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"]
Oct 29 23:35:44.458300 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:44.458274    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy" containerID="cri-o://b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834" gracePeriod=30
Oct 29 23:35:44.472238 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:44.472197    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"]
Oct 29 23:35:44.472454 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:44.472410    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" containerName="istio-proxy" containerID="cri-o://bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb" gracePeriod=30
Oct 29 23:35:47.162459 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:47.162423    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:35:47.602774 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:47.602683    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 29 23:35:47.602924 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:47.602780    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:35:49.045117 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.045071    6040 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 29 23:35:49.045514 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.045132    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:35:49.728848 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.728772    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:35:49.771702 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.771677    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:35:49.773679 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.773663    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:35:49.877717 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.877682    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-data\") pod \"22a149b5-4e2b-44b1-b454-5244a760273e\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") "
Oct 29 23:35:49.877935 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.877749    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-socket\") pod \"22a149b5-4e2b-44b1-b454-5244a760273e\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") "
Oct 29 23:35:49.877935 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.877789    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-envoy\") pod \"c4a1ac1b-341e-45da-9958-211394cd4483\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") "
Oct 29 23:35:49.877935 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.877829    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4a1ac1b-341e-45da-9958-211394cd4483-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4a1ac1b-341e-45da-9958-211394cd4483-istiod-ca-cert\") pod \"c4a1ac1b-341e-45da-9958-211394cd4483\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") "
Oct 29 23:35:49.877935 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.877875    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-certs\") pod \"9663e220-9cf3-4217-a320-9330dfaf1012\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") "
Oct 29 23:35:49.877935 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.877910    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9663e220-9cf3-4217-a320-9330dfaf1012-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663e220-9cf3-4217-a320-9330dfaf1012-istiod-ca-cert\") pod \"9663e220-9cf3-4217-a320-9330dfaf1012\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") "
Oct 29 23:35:49.878221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.877949    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-kube-api-access-t2hz6\" (UniqueName: \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-kube-api-access-t2hz6\") pod \"22a149b5-4e2b-44b1-b454-5244a760273e\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") "
Oct 29 23:35:49.878221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.877985    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-socket\") pod \"9663e220-9cf3-4217-a320-9330dfaf1012\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") "
Oct 29 23:35:49.878221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878003    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-data" pod "22a149b5-4e2b-44b1-b454-5244a760273e" (UID: "22a149b5-4e2b-44b1-b454-5244a760273e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.878221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878018    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-kube-api-access-cnchz\" (UniqueName: \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-kube-api-access-cnchz\") pod \"9663e220-9cf3-4217-a320-9330dfaf1012\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") "
Oct 29 23:35:49.878221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878052    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-istio-token\" (UniqueName: \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-istio-token\") pod \"9663e220-9cf3-4217-a320-9330dfaf1012\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") "
Oct 29 23:35:49.878221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878060    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-socket" pod "22a149b5-4e2b-44b1-b454-5244a760273e" (UID: "22a149b5-4e2b-44b1-b454-5244a760273e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.878221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878106    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/22a149b5-4e2b-44b1-b454-5244a760273e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22a149b5-4e2b-44b1-b454-5244a760273e-istio-podinfo\") pod \"22a149b5-4e2b-44b1-b454-5244a760273e\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") "
Oct 29 23:35:49.878221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878147    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-credential-socket\") pod \"22a149b5-4e2b-44b1-b454-5244a760273e\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") "
Oct 29 23:35:49.878221 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878180    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-data\") pod \"9663e220-9cf3-4217-a320-9330dfaf1012\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878226    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c4a1ac1b-341e-45da-9958-211394cd4483-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4a1ac1b-341e-45da-9958-211394cd4483-istio-podinfo\") pod \"c4a1ac1b-341e-45da-9958-211394cd4483\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878260    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-envoy\") pod \"22a149b5-4e2b-44b1-b454-5244a760273e\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878294    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22a149b5-4e2b-44b1-b454-5244a760273e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22a149b5-4e2b-44b1-b454-5244a760273e-istiod-ca-cert\") pod \"22a149b5-4e2b-44b1-b454-5244a760273e\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878325    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-certs\") pod \"22a149b5-4e2b-44b1-b454-5244a760273e\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878363    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-data\") pod \"c4a1ac1b-341e-45da-9958-211394cd4483\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878364    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-socket" pod "9663e220-9cf3-4217-a320-9330dfaf1012" (UID: "9663e220-9cf3-4217-a320-9330dfaf1012"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878385    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-certs" pod "9663e220-9cf3-4217-a320-9330dfaf1012" (UID: "9663e220-9cf3-4217-a320-9330dfaf1012"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878399    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-credential-socket\") pod \"c4a1ac1b-341e-45da-9958-211394cd4483\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878420    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9663e220-9cf3-4217-a320-9330dfaf1012-istiod-ca-cert" pod "9663e220-9cf3-4217-a320-9330dfaf1012" (UID: "9663e220-9cf3-4217-a320-9330dfaf1012"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878451    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-istio-token\" (UniqueName: \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-istio-token\") pod \"22a149b5-4e2b-44b1-b454-5244a760273e\" (UID: \"22a149b5-4e2b-44b1-b454-5244a760273e\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878501    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-socket\") pod \"c4a1ac1b-341e-45da-9958-211394cd4483\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878560    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-istio-token\" (UniqueName: \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-istio-token\") pod \"c4a1ac1b-341e-45da-9958-211394cd4483\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878600    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-certs\") pod \"c4a1ac1b-341e-45da-9958-211394cd4483\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878609    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-credential-socket" pod "c4a1ac1b-341e-45da-9958-211394cd4483" (UID: "c4a1ac1b-341e-45da-9958-211394cd4483"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878638    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9663e220-9cf3-4217-a320-9330dfaf1012-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663e220-9cf3-4217-a320-9330dfaf1012-istio-podinfo\") pod \"9663e220-9cf3-4217-a320-9330dfaf1012\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") "
Oct 29 23:35:49.878687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878670    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-credential-socket\") pod \"9663e220-9cf3-4217-a320-9330dfaf1012\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") "
Oct 29 23:35:49.879556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878699    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-envoy\") pod \"9663e220-9cf3-4217-a320-9330dfaf1012\" (UID: \"9663e220-9cf3-4217-a320-9330dfaf1012\") "
Oct 29 23:35:49.879556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878764    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-kube-api-access-hk2hw\" (UniqueName: \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-kube-api-access-hk2hw\") pod \"c4a1ac1b-341e-45da-9958-211394cd4483\" (UID: \"c4a1ac1b-341e-45da-9958-211394cd4483\") "
Oct 29 23:35:49.879556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.878961    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4a1ac1b-341e-45da-9958-211394cd4483-istiod-ca-cert" pod "c4a1ac1b-341e-45da-9958-211394cd4483" (UID: "c4a1ac1b-341e-45da-9958-211394cd4483"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:49.879556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.879223    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.879556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.879246    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.879556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.879262    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4a1ac1b-341e-45da-9958-211394cd4483-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.879556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.879275    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.879556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.879287    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663e220-9cf3-4217-a320-9330dfaf1012-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.879556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.879299    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.879556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.879311    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.880080 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.879595    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-data" pod "9663e220-9cf3-4217-a320-9330dfaf1012" (UID: "9663e220-9cf3-4217-a320-9330dfaf1012"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.880080 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.879918    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-credential-socket" pod "22a149b5-4e2b-44b1-b454-5244a760273e" (UID: "22a149b5-4e2b-44b1-b454-5244a760273e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.880229 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.880204    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22a149b5-4e2b-44b1-b454-5244a760273e-istiod-ca-cert" pod "22a149b5-4e2b-44b1-b454-5244a760273e" (UID: "22a149b5-4e2b-44b1-b454-5244a760273e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:35:49.881726 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.881299    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-credential-socket" pod "9663e220-9cf3-4217-a320-9330dfaf1012" (UID: "9663e220-9cf3-4217-a320-9330dfaf1012"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.881726 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.881368    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22a149b5-4e2b-44b1-b454-5244a760273e-istio-podinfo" pod "22a149b5-4e2b-44b1-b454-5244a760273e" (UID: "22a149b5-4e2b-44b1-b454-5244a760273e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:49.881726 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.881577    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-socket" pod "c4a1ac1b-341e-45da-9958-211394cd4483" (UID: "c4a1ac1b-341e-45da-9958-211394cd4483"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.882312 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.881977    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4a1ac1b-341e-45da-9958-211394cd4483-istio-podinfo" pod "c4a1ac1b-341e-45da-9958-211394cd4483" (UID: "c4a1ac1b-341e-45da-9958-211394cd4483"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:49.882312 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.882075    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-certs" pod "22a149b5-4e2b-44b1-b454-5244a760273e" (UID: "22a149b5-4e2b-44b1-b454-5244a760273e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.882312 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.882141    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-data" pod "c4a1ac1b-341e-45da-9958-211394cd4483" (UID: "c4a1ac1b-341e-45da-9958-211394cd4483"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.882312 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.882240    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-certs" pod "c4a1ac1b-341e-45da-9958-211394cd4483" (UID: "c4a1ac1b-341e-45da-9958-211394cd4483"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.882602 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.882577    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-kube-api-access-cnchz" pod "9663e220-9cf3-4217-a320-9330dfaf1012" (UID: "9663e220-9cf3-4217-a320-9330dfaf1012"). InnerVolumeSpecName "kube-api-access-cnchz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:49.887077 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.887052    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-envoy" pod "22a149b5-4e2b-44b1-b454-5244a760273e" (UID: "22a149b5-4e2b-44b1-b454-5244a760273e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.888826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.888799    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-kube-api-access-hk2hw" pod "c4a1ac1b-341e-45da-9958-211394cd4483" (UID: "c4a1ac1b-341e-45da-9958-211394cd4483"). InnerVolumeSpecName "kube-api-access-hk2hw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:49.889202 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.889160    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-kube-api-access-t2hz6" pod "22a149b5-4e2b-44b1-b454-5244a760273e" (UID: "22a149b5-4e2b-44b1-b454-5244a760273e"). InnerVolumeSpecName "kube-api-access-t2hz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:49.889202 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.889173    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-envoy" pod "9663e220-9cf3-4217-a320-9330dfaf1012" (UID: "9663e220-9cf3-4217-a320-9330dfaf1012"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.889446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.889420    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-istio-token" pod "9663e220-9cf3-4217-a320-9330dfaf1012" (UID: "9663e220-9cf3-4217-a320-9330dfaf1012"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:49.889446 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.889438    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9663e220-9cf3-4217-a320-9330dfaf1012-istio-podinfo" pod "9663e220-9cf3-4217-a320-9330dfaf1012" (UID: "9663e220-9cf3-4217-a320-9330dfaf1012"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:35:49.889571 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.889478    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-istio-token" pod "22a149b5-4e2b-44b1-b454-5244a760273e" (UID: "22a149b5-4e2b-44b1-b454-5244a760273e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:49.890196 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.890173    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-envoy" pod "c4a1ac1b-341e-45da-9958-211394cd4483" (UID: "c4a1ac1b-341e-45da-9958-211394cd4483"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:35:49.891370 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.891344    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-istio-token" pod "c4a1ac1b-341e-45da-9958-211394cd4483" (UID: "c4a1ac1b-341e-45da-9958-211394cd4483"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:35:49.980765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980685    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22a149b5-4e2b-44b1-b454-5244a760273e-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.980765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980712    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.980765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980722    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.980765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980731    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4a1ac1b-341e-45da-9958-211394cd4483-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.980765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980753    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.980765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980762    6040 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22a149b5-4e2b-44b1-b454-5244a760273e-istiod-ca-cert\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.980765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980769    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22a149b5-4e2b-44b1-b454-5244a760273e-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980777    6040 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-data\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980786    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980794    6040 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980803    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980823    6040 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-workload-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980831    6040 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663e220-9cf3-4217-a320-9330dfaf1012-istio-podinfo\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980838    6040 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-credential-socket\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980846    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663e220-9cf3-4217-a320-9330dfaf1012-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980855    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk2hw\" (UniqueName: \"kubernetes.io/projected/c4a1ac1b-341e-45da-9958-211394cd4483-kube-api-access-hk2hw\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980863    6040 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4a1ac1b-341e-45da-9958-211394cd4483-istio-envoy\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980870    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2hz6\" (UniqueName: \"kubernetes.io/projected/22a149b5-4e2b-44b1-b454-5244a760273e-kube-api-access-t2hz6\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980878    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnchz\" (UniqueName: \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-kube-api-access-cnchz\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:49.981078 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:49.980886    6040 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663e220-9cf3-4217-a320-9330dfaf1012-istio-token\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:35:50.285247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.285150    6040 generic.go:356] "Generic (PLEG): container finished" podID="9663e220-9cf3-4217-a320-9330dfaf1012" containerID="b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834" exitCode=0
Oct 29 23:35:50.285247 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.285201    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"
Oct 29 23:35:50.285803 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.285244    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" event={"ID":"9663e220-9cf3-4217-a320-9330dfaf1012","Type":"ContainerDied","Data":"b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834"}
Oct 29 23:35:50.285803 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.285308    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45" event={"ID":"9663e220-9cf3-4217-a320-9330dfaf1012","Type":"ContainerDied","Data":"cabb760b63c02688795dccf8a09766025cfe5f0f172a02c69ab40b3666f43c9d"}
Oct 29 23:35:50.285803 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.285341    6040 scope.go:122] "RemoveContainer" containerID="b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834"
Oct 29 23:35:50.287282 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.287235    6040 generic.go:356] "Generic (PLEG): container finished" podID="22a149b5-4e2b-44b1-b454-5244a760273e" containerID="bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb" exitCode=0
Oct 29 23:35:50.287394 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.287303    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"
Oct 29 23:35:50.287394 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.287337    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" event={"ID":"22a149b5-4e2b-44b1-b454-5244a760273e","Type":"ContainerDied","Data":"bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb"}
Oct 29 23:35:50.287394 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.287389    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6" event={"ID":"22a149b5-4e2b-44b1-b454-5244a760273e","Type":"ContainerDied","Data":"6563d1b90d3d445ae413a52ea1ea539c86da29ff3aa463921fffb348e3d4f535"}
Oct 29 23:35:50.289146 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.289122    6040 generic.go:356] "Generic (PLEG): container finished" podID="c4a1ac1b-341e-45da-9958-211394cd4483" containerID="e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64" exitCode=0
Oct 29 23:35:50.289265 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.289177    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" event={"ID":"c4a1ac1b-341e-45da-9958-211394cd4483","Type":"ContainerDied","Data":"e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64"}
Oct 29 23:35:50.289265 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.289215    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls" event={"ID":"c4a1ac1b-341e-45da-9958-211394cd4483","Type":"ContainerDied","Data":"9b26cc76d96d25990dd23a2d28e5b5feb4a476d770b052b5deb94132a7a25040"}
Oct 29 23:35:50.289378 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.289278    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"
Oct 29 23:35:50.301197 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.299937    6040 scope.go:122] "RemoveContainer" containerID="b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834"
Oct 29 23:35:50.301197 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:50.300363    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834\": container with ID starting with b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834 not found: ID does not exist" containerID="b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834"
Oct 29 23:35:50.301197 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.300390    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834"} err="failed to get container status \"b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834\": rpc error: code = NotFound desc = could not find container \"b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834\": container with ID starting with b1a1b1533e851625d85d2c45c3474fc0c9fd5302568a4700f8c5dff988f8f834 not found: ID does not exist"
Oct 29 23:35:50.301197 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.300411    6040 scope.go:122] "RemoveContainer" containerID="bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb"
Oct 29 23:35:50.311517 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.310272    6040 scope.go:122] "RemoveContainer" containerID="bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb"
Oct 29 23:35:50.311517 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:50.310554    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb\": container with ID starting with bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb not found: ID does not exist" containerID="bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb"
Oct 29 23:35:50.311517 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.310584    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb"} err="failed to get container status \"bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb\": rpc error: code = NotFound desc = could not find container \"bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb\": container with ID starting with bc968ab97214c9bd0cb5f2a37d4ad964c27ca7b610fc752447eb10c79e0d6bfb not found: ID does not exist"
Oct 29 23:35:50.311517 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.310609    6040 scope.go:122] "RemoveContainer" containerID="e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64"
Oct 29 23:35:50.321958 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.321937    6040 scope.go:122] "RemoveContainer" containerID="e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64"
Oct 29 23:35:50.322304 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:50.322266    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64\": container with ID starting with e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64 not found: ID does not exist" containerID="e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64"
Oct 29 23:35:50.322423 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.322300    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64"} err="failed to get container status \"e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64\": rpc error: code = NotFound desc = could not find container \"e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64\": container with ID starting with e80c4260baf79af407bef90bd3a3239229c49edd0df056df3bee3410bcac3f64 not found: ID does not exist"
Oct 29 23:35:50.359096 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.359060    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"]
Oct 29 23:35:50.369036 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.369007    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-jp6d6"]
Oct 29 23:35:50.398494 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.398458    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"]
Oct 29 23:35:50.405717 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.405682    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-lcx45"]
Oct 29 23:35:50.426442 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.426408    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" path="/var/lib/kubelet/pods/22a149b5-4e2b-44b1-b454-5244a760273e/volumes"
Oct 29 23:35:50.427082 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.427059    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" path="/var/lib/kubelet/pods/9663e220-9cf3-4217-a320-9330dfaf1012/volumes"
Oct 29 23:35:50.439886 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.439862    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"]
Oct 29 23:35:50.446979 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:50.446954    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-xcdls"]
Oct 29 23:35:52.453083 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:35:52.453045    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" path="/var/lib/kubelet/pods/c4a1ac1b-341e-45da-9958-211394cd4483/volumes"
Oct 29 23:35:59.778758 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:35:59.778710    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:36:29.782989 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:36:29.782948    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:36:40.947972 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:36:40.947939    6040 scope.go:122] "RemoveContainer" containerID="8f2deccd32e43412a6418c6910d77f7babe322cf1ede1a7fb1cba782e43ac075"
Oct 29 23:36:40.957188 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:36:40.957167    6040 scope.go:122] "RemoveContainer" containerID="6427fca2af1ff5667e1e1671c998dbf7a67532e2c32a2da1fee5652a8e7cd802"
Oct 29 23:36:57.028210 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:36:57.028168    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"]
Oct 29 23:36:57.039733 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:36:57.039697    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0e30e0cc-tfd5c"]
Oct 29 23:36:58.423267 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:36:58.423238    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6e92817-391d-4736-ab23-0de8643a7bbc" path="/var/lib/kubelet/pods/e6e92817-391d-4736-ab23-0de8643a7bbc/volumes"
Oct 29 23:36:59.365419 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:36:59.365384    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:36:59.785765 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:36:59.785702    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:37:02.139762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:02.138922    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-lbpk9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:37:02.139762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:02.139033    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" podUID="3eaa91d2-99cc-4e70-b98f-0c5a851dfb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:37:07.028684 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:07.028641    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"]
Oct 29 23:37:07.040028 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:07.040001    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4f8c666-l499q"]
Oct 29 23:37:08.424859 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:08.424816    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac38c214-7803-4b30-ba48-76dd80495c11" path="/var/lib/kubelet/pods/ac38c214-7803-4b30-ba48-76dd80495c11/volumes"
Oct 29 23:37:16.487493 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:16.487457    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:37:21.092390 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:21.092356    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:37:23.037760 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:23.037639    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"]
Oct 29 23:37:23.054033 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:23.054000    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-46c8d7a9-nfzpj"]
Oct 29 23:37:24.424475 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:24.424442    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a7a0d36-7039-4782-9835-c159ac1e836e" path="/var/lib/kubelet/pods/3a7a0d36-7039-4782-9835-c159ac1e836e/volumes"
Oct 29 23:37:29.778134 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:37:29.778099    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:37:39.357555 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:39.357518    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:37:55.069045 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:37:55.068965    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:37:59.779145 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:37:59.779101    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:38:08.027984 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:38:08.027948    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"]
Oct 29 23:38:08.039574 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:38:08.039546    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e41943f9-772kz"]
Oct 29 23:38:08.425008 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:38:08.424972    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c22b1162-a7c9-49ed-9426-1a8804fb323f" path="/var/lib/kubelet/pods/c22b1162-a7c9-49ed-9426-1a8804fb323f/volumes"
Oct 29 23:38:18.406912 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:38:18.406868    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:38:28.695392 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:38:28.695357    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:38:29.779251 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:38:29.779218    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:38:33.181085 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:38:33.181045    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:38:41.001268 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:38:41.001236    6040 scope.go:122] "RemoveContainer" containerID="91ffb3f54ef7003b9a11f944e32caffe78177f7261ee56d1afd41288ac0f618f"
Oct 29 23:38:41.009889 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:38:41.009871    6040 scope.go:122] "RemoveContainer" containerID="fd200a663770d07bce988005c29979bd42acdb114ff03e65320d053200b3a166"
Oct 29 23:38:41.017656 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:38:41.017633    6040 scope.go:122] "RemoveContainer" containerID="14a47f68187835a4bccb7588bf9c37c755d08c9af459445ff969713197cc4c1e"
Oct 29 23:38:53.575443 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:38:53.575404    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:38:58.944316 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:38:58.941362    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:38:59.779076 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:38:59.779033    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:39:15.089733 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:39:15.089696    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:39:19.606151 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:39:19.606111    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:39:20.027687 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:39:20.027655    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"]
Oct 29 23:39:20.039400 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:39:20.039370    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ab06168f-2qw2h"]
Oct 29 23:39:20.424560 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:39:20.424524    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51061993-4d39-4fab-9f1d-ba0222fa7e6c" path="/var/lib/kubelet/pods/51061993-4d39-4fab-9f1d-ba0222fa7e6c/volumes"
Oct 29 23:39:29.778268 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:39:29.778226    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:39:36.040290 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:39:36.040252    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"]
Oct 29 23:39:36.053016 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:39:36.052983    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e30686a-4zpwl"]
Oct 29 23:39:36.423468 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:39:36.423436    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ccb7ea93-728f-46a0-a1b0-1358aab430ee" path="/var/lib/kubelet/pods/ccb7ea93-728f-46a0-a1b0-1358aab430ee/volumes"
Oct 29 23:39:39.195571 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:39:39.195534    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:39:41.064757 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:39:41.064710    6040 scope.go:122] "RemoveContainer" containerID="d888c82fbd67905327ac4a5d3bd9325b9328a3856fde6a60c7e2c788321c9f6b"
Oct 29 23:39:55.103518 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:39:55.103475    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:39:59.779615 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:39:59.779581    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:40:03.028667 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:03.028629    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"]
Oct 29 23:40:03.040512 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:03.040482    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b4dbfcf-zl95g"]
Oct 29 23:40:04.423963 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:04.423930    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a8fbb40-37c6-44ef-8339-5bd0187e687b" path="/var/lib/kubelet/pods/4a8fbb40-37c6-44ef-8339-5bd0187e687b/volumes"
Oct 29 23:40:06.775305 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:06.775273    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:40:12.809603 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:12.809570    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:40:27.972898 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:27.972865    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:40:28.033799 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:28.033765    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"]
Oct 29 23:40:28.045761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:28.045723    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-60cead78-h9ktd"]
Oct 29 23:40:28.425474 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:28.425436    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab380e45-5b8c-437f-a387-80d5194066d2" path="/var/lib/kubelet/pods/ab380e45-5b8c-437f-a387-80d5194066d2/volumes"
Oct 29 23:40:29.778155 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:40:29.778120    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:40:33.180652 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:33.180618    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:40:41.095913 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:41.095881    6040 scope.go:122] "RemoveContainer" containerID="9149bb6d1b243e805f95dc72c19abcd7ebcb407c7cac0c7816deecba3be590e3"
Oct 29 23:40:41.104168 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:41.104149    6040 scope.go:122] "RemoveContainer" containerID="fa256f9ce222adfb9d6aabbbf0ad2901e99e4cd9f2467e59f48af66e493f8754"
Oct 29 23:40:51.567546 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:51.567463    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:40:56.606358 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:40:56.606323    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:40:59.779592 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:40:59.779556    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:41:09.981464 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:41:09.981425    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:41:24.875408 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:41:24.875366    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:41:29.780200 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:41:29.780157    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:41:39.711057 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:41:39.711022    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:41:41.127844 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:41:41.127815    6040 scope.go:122] "RemoveContainer" containerID="24d890d55c6df4f05240553cba5f7478f883c6c321f156b7b0862e83fb924cca"
Oct 29 23:41:41.136009 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:41:41.135988    6040 scope.go:122] "RemoveContainer" containerID="2686c919f8c193d9413175e91fab970c06427a34e91bc86bc484f7c993dd4ad2"
Oct 29 23:41:57.168892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:41:57.168853    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:41:59.778915 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:41:59.778870    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:42:11.500366 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:42:11.500325    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:42:29.779172 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:42:29.779132    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:42:30.075398 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:42:30.075309    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:42:32.156762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:42:32.137882    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-lbpk9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:42:32.156762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:42:32.137965    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-lbpk9" podUID="3eaa91d2-99cc-4e70-b98f-0c5a851dfb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:42:32.168992 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:42:32.168928    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-pb266 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:42:32.169569 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:42:32.169527    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-pb266" podUID="41115057-f034-42e6-96c0-1b398529a14c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:42:35.286126 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:42:35.286092    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:42:59.778035 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:42:59.778000    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:43:29.778211 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:43:29.778173    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:43:59.778651 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:43:59.778563    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:44:00.990922 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:44:00.990874    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:44:15.473710 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:44:15.473667    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:44:29.778788 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:44:29.778748    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:44:41.183597 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:44:41.183564    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:44:56.193802 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:44:56.193766    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:44:59.779021 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:44:59.778984    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:45:00.149021 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.148986    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"]
Oct 29 23:45:00.149414 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149396    6040 cpu_manager.go:414] "containerMap: removing container" podUID="ab380e45-5b8c-437f-a387-80d5194066d2" containerName="revoke-keys"
Oct 29 23:45:00.149500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149416    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab380e45-5b8c-437f-a387-80d5194066d2" containerName="revoke-keys"
Oct 29 23:45:00.149500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149449    6040 cpu_manager.go:414] "containerMap: removing container" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main"
Oct 29 23:45:00.149500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149457    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main"
Oct 29 23:45:00.149500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149467    6040 cpu_manager.go:414] "containerMap: removing container" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" containerName="istio-proxy"
Oct 29 23:45:00.149500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149475    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" containerName="istio-proxy"
Oct 29 23:45:00.149500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149488    6040 cpu_manager.go:414] "containerMap: removing container" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy"
Oct 29 23:45:00.149500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149496    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149507    6040 cpu_manager.go:414] "containerMap: removing container" podUID="186078cc-b997-4651-9dcf-b33603f13047" containerName="main"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149516    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="186078cc-b997-4651-9dcf-b33603f13047" containerName="main"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149526    6040 cpu_manager.go:414] "containerMap: removing container" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149534    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149544    6040 cpu_manager.go:414] "containerMap: removing container" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149551    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149564    6040 cpu_manager.go:414] "containerMap: removing container" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149572    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149584    6040 cpu_manager.go:414] "containerMap: removing container" podUID="7bbff9d3-81f5-4f3f-9be1-afba3a8058ef" containerName="payload-pre-processing"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149594    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bbff9d3-81f5-4f3f-9be1-afba3a8058ef" containerName="payload-pre-processing"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149604    6040 cpu_manager.go:414] "containerMap: removing container" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149611    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149620    6040 cpu_manager.go:414] "containerMap: removing container" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149628    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149643    6040 cpu_manager.go:414] "containerMap: removing container" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149650    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149759    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0d3ee3e-07bd-4868-a39a-dccab1fc6222" containerName="main"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149773    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="22a149b5-4e2b-44b1-b454-5244a760273e" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149782    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="9663e220-9cf3-4217-a320-9330dfaf1012" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149792    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="186078cc-b997-4651-9dcf-b33603f13047" containerName="main"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149800    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="a292574a-72a5-427c-a344-255fac8aa567" containerName="main"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149807    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="256870d0-9dce-4e09-8374-9835d54c4f78" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149816    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d9dcc91-f5e8-47fb-801e-251be89b55d5" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149825    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bbff9d3-81f5-4f3f-9be1-afba3a8058ef" containerName="payload-pre-processing"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149833    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc61fe28-ed42-4655-99de-d63d17a30a3f" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149840    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc92da1f-4308-4050-8c1f-7a926c5eae0f" containerName="main"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149849    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4a1ac1b-341e-45da-9958-211394cd4483" containerName="istio-proxy"
Oct 29 23:45:00.149870 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.149860    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab380e45-5b8c-437f-a387-80d5194066d2" containerName="revoke-keys"
Oct 29 23:45:00.154692 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.154669    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"
Oct 29 23:45:00.175351 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.175327    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"]
Oct 29 23:45:00.181839 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.181819    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:45:00.182159 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.182136    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:45:00.182268 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.182207    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:45:00.264976 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.264946    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff6w2\" (UniqueName: \"kubernetes.io/projected/471dc31f-b32c-44ec-8922-cd68dc8c31c6-kube-api-access-ff6w2\") pod \"maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9\" (UID: \"471dc31f-b32c-44ec-8922-cd68dc8c31c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"
Oct 29 23:45:00.265160 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.265012    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/471dc31f-b32c-44ec-8922-cd68dc8c31c6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9\" (UID: \"471dc31f-b32c-44ec-8922-cd68dc8c31c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"
Oct 29 23:45:00.365970 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.365938    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/471dc31f-b32c-44ec-8922-cd68dc8c31c6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9\" (UID: \"471dc31f-b32c-44ec-8922-cd68dc8c31c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"
Oct 29 23:45:00.366129 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.366011    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff6w2\" (UniqueName: \"kubernetes.io/projected/471dc31f-b32c-44ec-8922-cd68dc8c31c6-kube-api-access-ff6w2\") pod \"maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9\" (UID: \"471dc31f-b32c-44ec-8922-cd68dc8c31c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"
Oct 29 23:45:00.366540 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.366521    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/471dc31f-b32c-44ec-8922-cd68dc8c31c6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9\" (UID: \"471dc31f-b32c-44ec-8922-cd68dc8c31c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"
Oct 29 23:45:00.389559 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.389529    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff6w2\" (UniqueName: \"kubernetes.io/projected/471dc31f-b32c-44ec-8922-cd68dc8c31c6-kube-api-access-ff6w2\") pod \"maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9\" (UID: \"471dc31f-b32c-44ec-8922-cd68dc8c31c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"
Oct 29 23:45:00.469271 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.469200    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"
Oct 29 23:45:00.766914 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:00.766884    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"]
Oct 29 23:45:00.769001 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:45:00.768970    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod471dc31f_b32c_44ec_8922_cd68dc8c31c6.slice/crio-b13751e62c5bdf8fbdcc1185dd4048fc2d4fd54541811acacd3f3a8e2a5574b8 WatchSource:0}: Error finding container b13751e62c5bdf8fbdcc1185dd4048fc2d4fd54541811acacd3f3a8e2a5574b8: Status 404 returned error can't find the container with id b13751e62c5bdf8fbdcc1185dd4048fc2d4fd54541811acacd3f3a8e2a5574b8
Oct 29 23:45:01.368708 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:01.368629    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9" event={"ID":"471dc31f-b32c-44ec-8922-cd68dc8c31c6","Type":"ContainerStarted","Data":"d4b71d221a6ef907eefbe6344e9ff01604ae8749e54c1f96c83ea8de959c3f58"}
Oct 29 23:45:01.368708 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:01.368676    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9" event={"ID":"471dc31f-b32c-44ec-8922-cd68dc8c31c6","Type":"ContainerStarted","Data":"b13751e62c5bdf8fbdcc1185dd4048fc2d4fd54541811acacd3f3a8e2a5574b8"}
Oct 29 23:45:01.408888 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:01.408825    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9" podStartSLOduration=1.408807286 podStartE2EDuration="1.408807286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:45:01.407359247 +0000 UTC m=+1881.102468125" watchObservedRunningTime="2026-10-29 23:45:01.408807286 +0000 UTC m=+1881.103916164"
Oct 29 23:45:02.375351 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:02.375312    6040 generic.go:356] "Generic (PLEG): container finished" podID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerID="d4b71d221a6ef907eefbe6344e9ff01604ae8749e54c1f96c83ea8de959c3f58" exitCode=60
Oct 29 23:45:02.375833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:02.375407    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9" event={"ID":"471dc31f-b32c-44ec-8922-cd68dc8c31c6","Type":"ContainerDied","Data":"d4b71d221a6ef907eefbe6344e9ff01604ae8749e54c1f96c83ea8de959c3f58"}
Oct 29 23:45:02.375833 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:02.375587    6040 scope.go:122] "RemoveContainer" containerID="d4b71d221a6ef907eefbe6344e9ff01604ae8749e54c1f96c83ea8de959c3f58"
Oct 29 23:45:03.382163 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:03.382131    6040 generic.go:356] "Generic (PLEG): container finished" podID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerID="af9df4542bb45aa364708d903d23874d1716e6c40c6d1a538488924101092505" exitCode=60
Oct 29 23:45:03.382558 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:03.382219    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9" event={"ID":"471dc31f-b32c-44ec-8922-cd68dc8c31c6","Type":"ContainerDied","Data":"af9df4542bb45aa364708d903d23874d1716e6c40c6d1a538488924101092505"}
Oct 29 23:45:03.382558 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:03.382276    6040 scope.go:122] "RemoveContainer" containerID="d4b71d221a6ef907eefbe6344e9ff01604ae8749e54c1f96c83ea8de959c3f58"
Oct 29 23:45:03.382558 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:03.382503    6040 scope.go:122] "RemoveContainer" containerID="af9df4542bb45aa364708d903d23874d1716e6c40c6d1a538488924101092505"
Oct 29 23:45:03.382774 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:45:03.382757    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9_odh-ai-gateway-infra(471dc31f-b32c-44ec-8922-cd68dc8c31c6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6"
Oct 29 23:45:04.388511 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:04.388486    6040 scope.go:122] "RemoveContainer" containerID="af9df4542bb45aa364708d903d23874d1716e6c40c6d1a538488924101092505"
Oct 29 23:45:04.388906 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:45:04.388698    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9_odh-ai-gateway-infra(471dc31f-b32c-44ec-8922-cd68dc8c31c6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6"
Oct 29 23:45:10.690368 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:10.690331    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:45:17.419142 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:17.419105    6040 scope.go:122] "RemoveContainer" containerID="af9df4542bb45aa364708d903d23874d1716e6c40c6d1a538488924101092505"
Oct 29 23:45:18.459325 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:18.459291    6040 generic.go:356] "Generic (PLEG): container finished" podID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerID="011a8b54b55a9d320cdbb66b08e3d6eb33749c6f1026cb9adec26862c637501a" exitCode=60
Oct 29 23:45:18.459817 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:18.459360    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9" event={"ID":"471dc31f-b32c-44ec-8922-cd68dc8c31c6","Type":"ContainerDied","Data":"011a8b54b55a9d320cdbb66b08e3d6eb33749c6f1026cb9adec26862c637501a"}
Oct 29 23:45:18.459817 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:18.459413    6040 scope.go:122] "RemoveContainer" containerID="af9df4542bb45aa364708d903d23874d1716e6c40c6d1a538488924101092505"
Oct 29 23:45:18.459946 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:18.459816    6040 scope.go:122] "RemoveContainer" containerID="011a8b54b55a9d320cdbb66b08e3d6eb33749c6f1026cb9adec26862c637501a"
Oct 29 23:45:18.460079 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:45:18.460055    6040 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9_odh-ai-gateway-infra(471dc31f-b32c-44ec-8922-cd68dc8c31c6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6"
Oct 29 23:45:19.510045 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:19.510011    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"]
Oct 29 23:45:19.649614 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:19.649591    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"
Oct 29 23:45:19.744887 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:19.744857    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/471dc31f-b32c-44ec-8922-cd68dc8c31c6-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/471dc31f-b32c-44ec-8922-cd68dc8c31c6-maas-api-service-ca\") pod \"471dc31f-b32c-44ec-8922-cd68dc8c31c6\" (UID: \"471dc31f-b32c-44ec-8922-cd68dc8c31c6\") "
Oct 29 23:45:19.745040 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:19.744899    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/471dc31f-b32c-44ec-8922-cd68dc8c31c6-kube-api-access-ff6w2\" (UniqueName: \"kubernetes.io/projected/471dc31f-b32c-44ec-8922-cd68dc8c31c6-kube-api-access-ff6w2\") pod \"471dc31f-b32c-44ec-8922-cd68dc8c31c6\" (UID: \"471dc31f-b32c-44ec-8922-cd68dc8c31c6\") "
Oct 29 23:45:19.745291 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:19.745151    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/471dc31f-b32c-44ec-8922-cd68dc8c31c6-maas-api-service-ca" pod "471dc31f-b32c-44ec-8922-cd68dc8c31c6" (UID: "471dc31f-b32c-44ec-8922-cd68dc8c31c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:45:19.746913 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:19.746885    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/471dc31f-b32c-44ec-8922-cd68dc8c31c6-kube-api-access-ff6w2" pod "471dc31f-b32c-44ec-8922-cd68dc8c31c6" (UID: "471dc31f-b32c-44ec-8922-cd68dc8c31c6"). InnerVolumeSpecName "kube-api-access-ff6w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:45:19.846136 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:19.846048    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/471dc31f-b32c-44ec-8922-cd68dc8c31c6-maas-api-service-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:45:19.846136 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:19.846082    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff6w2\" (UniqueName: \"kubernetes.io/projected/471dc31f-b32c-44ec-8922-cd68dc8c31c6-kube-api-access-ff6w2\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:45:20.472565 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:20.472538    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"
Oct 29 23:45:20.472722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:20.472560    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9" event={"ID":"471dc31f-b32c-44ec-8922-cd68dc8c31c6","Type":"ContainerDied","Data":"b13751e62c5bdf8fbdcc1185dd4048fc2d4fd54541811acacd3f3a8e2a5574b8"}
Oct 29 23:45:20.472722 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:20.472612    6040 scope.go:122] "RemoveContainer" containerID="011a8b54b55a9d320cdbb66b08e3d6eb33749c6f1026cb9adec26862c637501a"
Oct 29 23:45:20.525438 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:20.525408    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"]
Oct 29 23:45:20.537175 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:20.537082    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-06f91a-2988862jt4t9"]
Oct 29 23:45:22.423456 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:22.423427    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" path="/var/lib/kubelet/pods/471dc31f-b32c-44ec-8922-cd68dc8c31c6/volumes"
Oct 29 23:45:24.230568 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:24.230535    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:45:29.779421 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:45:29.779388    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:45:37.364265 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:37.364229    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:45:49.392305 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:45:49.392272    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:45:59.778817 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:45:59.778783    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:46:05.783349 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:46:05.783311    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:46:26.917427 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:46:26.917392    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:46:29.795517 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:46:29.795483    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:46:38.891900 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:46:38.891865    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:46:44.577481 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:46:44.577446    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:46:46.235942 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:46:46.235903    6040 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-dh2pt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:46:46.236648 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:46:46.236605    6040 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-dh2pt" podUID="07f389b2-8dd8-4149-ad51-bacbf9f1adbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:46:50.505889 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:46:50.505159    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:46:58.899495 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:46:58.899461    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:46:59.778775 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:46:59.778709    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:47:03.179101 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:03.179068    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:47:18.274327 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:18.274292    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:47:19.391084 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.391045    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"]
Oct 29 23:47:19.391526 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.391416    6040 cpu_manager.go:414] "containerMap: removing container" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerName="cleanup"
Oct 29 23:47:19.391526 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.391426    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerName="cleanup"
Oct 29 23:47:19.391526 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.391460    6040 cpu_manager.go:414] "containerMap: removing container" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerName="cleanup"
Oct 29 23:47:19.391526 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.391466    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerName="cleanup"
Oct 29 23:47:19.391712 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.391543    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerName="cleanup"
Oct 29 23:47:19.391712 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.391555    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerName="cleanup"
Oct 29 23:47:19.394794 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.394775    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.423579 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.423545    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:47:19.423771 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.423620    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:47:19.423853 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.423782    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 29 23:47:19.426508 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.426487    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-k7tlz\""
Oct 29 23:47:19.426629 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.426543    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 29 23:47:19.426761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.426722    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"]
Oct 29 23:47:19.502337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.502304    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-openshift-service-ca\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.502519 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.502353    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-maas-api-tls\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.502519 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.502477    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7rqx\" (UniqueName: \"kubernetes.io/projected/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-kube-api-access-p7rqx\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.502646 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.502516    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-metrics-certs\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.603625 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.603595    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-openshift-service-ca\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.603842 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.603631    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-maas-api-tls\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.603842 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.603659    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7rqx\" (UniqueName: \"kubernetes.io/projected/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-kube-api-access-p7rqx\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.603842 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.603677    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-metrics-certs\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.604259 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.604235    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-openshift-service-ca\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.606084 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.606061    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-metrics-certs\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.606190 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.606168    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-maas-api-tls\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.649282 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.649217    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7rqx\" (UniqueName: \"kubernetes.io/projected/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-kube-api-access-p7rqx\") pod \"maas-api-5c48877c75-9pht4\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:19.708021 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:19.707989    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:20.009111 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:20.009088    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"]
Oct 29 23:47:20.010972 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:47:20.010936    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845d1368_2ee1_4ca2_91aa_cf60ebc1df35.slice/crio-2fcc244727408140dbef9586c46231cefa862553f810ffc050f818af0cb56bbf WatchSource:0}: Error finding container 2fcc244727408140dbef9586c46231cefa862553f810ffc050f818af0cb56bbf: Status 404 returned error can't find the container with id 2fcc244727408140dbef9586c46231cefa862553f810ffc050f818af0cb56bbf
Oct 29 23:47:20.012293 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:20.012273    6040 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 29 23:47:20.248813 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:20.248777    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4" event={"ID":"845d1368-2ee1-4ca2-91aa-cf60ebc1df35","Type":"ContainerStarted","Data":"2fcc244727408140dbef9586c46231cefa862553f810ffc050f818af0cb56bbf"}
Oct 29 23:47:21.255379 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:21.255346    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4" event={"ID":"845d1368-2ee1-4ca2-91aa-cf60ebc1df35","Type":"ContainerStarted","Data":"5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438"}
Oct 29 23:47:21.255850 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:21.255465    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:21.298039 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:21.297990    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4" podStartSLOduration=1.829565092 podStartE2EDuration="2.297976726s" totalImagesPullingTime="468.411634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:47:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:47:21.296169636 +0000 UTC m=+2020.991278516" watchObservedRunningTime="2026-10-29 23:47:21.297976726 +0000 UTC m=+2020.993085605"
Oct 29 23:47:27.267662 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:27.267629    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:27.895329 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:27.895292    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:47:29.794614 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:47:29.794578    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:47:35.977489 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:35.977457    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:47:39.056910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:39.056878    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"]
Oct 29 23:47:39.057414 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:39.057096    6040 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4" podUID="845d1368-2ee1-4ca2-91aa-cf60ebc1df35" containerName="maas-api" containerID="cri-o://5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438" gracePeriod=30
Oct 29 23:47:40.308518 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.308492    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:40.382898 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.382855    6040 generic.go:356] "Generic (PLEG): container finished" podID="845d1368-2ee1-4ca2-91aa-cf60ebc1df35" containerID="5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438" exitCode=0
Oct 29 23:47:40.383116 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.382914    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"
Oct 29 23:47:40.383116 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.382967    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4" event={"ID":"845d1368-2ee1-4ca2-91aa-cf60ebc1df35","Type":"ContainerDied","Data":"5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438"}
Oct 29 23:47:40.383116 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.383030    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-9pht4" event={"ID":"845d1368-2ee1-4ca2-91aa-cf60ebc1df35","Type":"ContainerDied","Data":"2fcc244727408140dbef9586c46231cefa862553f810ffc050f818af0cb56bbf"}
Oct 29 23:47:40.383116 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.383057    6040 scope.go:122] "RemoveContainer" containerID="5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438"
Oct 29 23:47:40.388423 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.388399    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-openshift-service-ca\") pod \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") "
Oct 29 23:47:40.388567 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.388493    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-maas-api-tls\") pod \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") "
Oct 29 23:47:40.388618 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.388565    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-metrics-certs\" (UniqueName: \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-metrics-certs\") pod \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") "
Oct 29 23:47:40.388670 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.388614    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-kube-api-access-p7rqx\" (UniqueName: \"kubernetes.io/projected/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-kube-api-access-p7rqx\") pod \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\" (UID: \"845d1368-2ee1-4ca2-91aa-cf60ebc1df35\") "
Oct 29 23:47:40.388881 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.388805    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-openshift-service-ca" pod "845d1368-2ee1-4ca2-91aa-cf60ebc1df35" (UID: "845d1368-2ee1-4ca2-91aa-cf60ebc1df35"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:47:40.389110 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.389078    6040 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-openshift-service-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:47:40.390758 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.390719    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-metrics-certs" pod "845d1368-2ee1-4ca2-91aa-cf60ebc1df35" (UID: "845d1368-2ee1-4ca2-91aa-cf60ebc1df35"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:47:40.390964 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.390922    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-maas-api-tls" pod "845d1368-2ee1-4ca2-91aa-cf60ebc1df35" (UID: "845d1368-2ee1-4ca2-91aa-cf60ebc1df35"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:47:40.391057 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.391037    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-kube-api-access-p7rqx" pod "845d1368-2ee1-4ca2-91aa-cf60ebc1df35" (UID: "845d1368-2ee1-4ca2-91aa-cf60ebc1df35"). InnerVolumeSpecName "kube-api-access-p7rqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:47:40.392112 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.392084    6040 scope.go:122] "RemoveContainer" containerID="5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438"
Oct 29 23:47:40.392390 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:47:40.392372    6040 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438\": container with ID starting with 5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438 not found: ID does not exist" containerID="5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438"
Oct 29 23:47:40.392456 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.392405    6040 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438"} err="failed to get container status \"5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438\": rpc error: code = NotFound desc = could not find container \"5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438\": container with ID starting with 5229c92000719851cc7df9d1e678ea708a6602c00d4e0c80d191e9413fa0b438 not found: ID does not exist"
Oct 29 23:47:40.490370 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.490341    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-maas-api-tls\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:47:40.490370 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.490365    6040 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-metrics-certs\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:47:40.490370 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.490375    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7rqx\" (UniqueName: \"kubernetes.io/projected/845d1368-2ee1-4ca2-91aa-cf60ebc1df35-kube-api-access-p7rqx\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:47:40.722570 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.722494    6040 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"]
Oct 29 23:47:40.733793 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:40.733765    6040 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-9pht4"]
Oct 29 23:47:42.423826 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.423792    6040 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="845d1368-2ee1-4ca2-91aa-cf60ebc1df35" path="/var/lib/kubelet/pods/845d1368-2ee1-4ca2-91aa-cf60ebc1df35/volumes"
Oct 29 23:47:42.822412 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.822335    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"]
Oct 29 23:47:42.822707 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.822695    6040 cpu_manager.go:414] "containerMap: removing container" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerName="cleanup"
Oct 29 23:47:42.822765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.822709    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerName="cleanup"
Oct 29 23:47:42.822765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.822720    6040 cpu_manager.go:414] "containerMap: removing container" podUID="845d1368-2ee1-4ca2-91aa-cf60ebc1df35" containerName="maas-api"
Oct 29 23:47:42.822765 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.822725    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="845d1368-2ee1-4ca2-91aa-cf60ebc1df35" containerName="maas-api"
Oct 29 23:47:42.822863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.822830    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="845d1368-2ee1-4ca2-91aa-cf60ebc1df35" containerName="maas-api"
Oct 29 23:47:42.822863 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.822843    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="471dc31f-b32c-44ec-8922-cd68dc8c31c6" containerName="cleanup"
Oct 29 23:47:42.827219 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.827204    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:42.843998 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.843975    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:47:42.846762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.846721    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"]
Oct 29 23:47:42.855931 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.855908    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:47:42.856034 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.855928    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:47:42.856034 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.855939    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:47:42.910235 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.910210    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-f84xb\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:42.910374 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.910261    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-f84xb\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:42.910440 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:42.910381    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt74d\" (UniqueName: \"kubernetes.io/projected/e6124ae8-fca1-4a31-ba83-145b7027f136-kube-api-access-zt74d\") pod \"maas-api-revoke-sub-f03629db-f84xb\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:43.011549 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:43.011519    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-f84xb\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:43.011706 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:43.011561    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-f84xb\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:43.011706 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:43.011688    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt74d\" (UniqueName: \"kubernetes.io/projected/e6124ae8-fca1-4a31-ba83-145b7027f136-kube-api-access-zt74d\") pod \"maas-api-revoke-sub-f03629db-f84xb\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:43.012191 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:43.012173    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-f84xb\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:43.013905 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:43.013888    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-f84xb\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:43.034344 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:43.034317    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt74d\" (UniqueName: \"kubernetes.io/projected/e6124ae8-fca1-4a31-ba83-145b7027f136-kube-api-access-zt74d\") pod \"maas-api-revoke-sub-f03629db-f84xb\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:43.139236 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:43.139209    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:43.436117 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:43.436090    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"]
Oct 29 23:47:43.437804 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:47:43.437778    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6124ae8_fca1_4a31_ba83_145b7027f136.slice/crio-badfa6ced1145a870e763d0956ff86edf7e8fb854f2ec48890d749a08cdb2da4 WatchSource:0}: Error finding container badfa6ced1145a870e763d0956ff86edf7e8fb854f2ec48890d749a08cdb2da4: Status 404 returned error can't find the container with id badfa6ced1145a870e763d0956ff86edf7e8fb854f2ec48890d749a08cdb2da4
Oct 29 23:47:44.404316 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:44.404285    6040 generic.go:356] "Generic (PLEG): container finished" podID="e6124ae8-fca1-4a31-ba83-145b7027f136" containerID="8a03219168e35c264875ce1355df7f7999caaac0056a2661f7cfcbbb313118c3" exitCode=0
Oct 29 23:47:44.404496 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:44.404369    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb" event={"ID":"e6124ae8-fca1-4a31-ba83-145b7027f136","Type":"ContainerDied","Data":"8a03219168e35c264875ce1355df7f7999caaac0056a2661f7cfcbbb313118c3"}
Oct 29 23:47:44.404496 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:44.404422    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb" event={"ID":"e6124ae8-fca1-4a31-ba83-145b7027f136","Type":"ContainerStarted","Data":"badfa6ced1145a870e763d0956ff86edf7e8fb854f2ec48890d749a08cdb2da4"}
Oct 29 23:47:45.538683 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:45.538657    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:45.637720 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:45.637683    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-openshift-ca\") pod \"e6124ae8-fca1-4a31-ba83-145b7027f136\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") "
Oct 29 23:47:45.637905 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:45.637729    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6124ae8-fca1-4a31-ba83-145b7027f136-kube-api-access-zt74d\" (UniqueName: \"kubernetes.io/projected/e6124ae8-fca1-4a31-ba83-145b7027f136-kube-api-access-zt74d\") pod \"e6124ae8-fca1-4a31-ba83-145b7027f136\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") "
Oct 29 23:47:45.637905 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:45.637872    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-xks-ca\") pod \"e6124ae8-fca1-4a31-ba83-145b7027f136\" (UID: \"e6124ae8-fca1-4a31-ba83-145b7027f136\") "
Oct 29 23:47:45.638124 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:45.638101    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-openshift-ca" pod "e6124ae8-fca1-4a31-ba83-145b7027f136" (UID: "e6124ae8-fca1-4a31-ba83-145b7027f136"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:47:45.639986 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:45.639962    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6124ae8-fca1-4a31-ba83-145b7027f136-kube-api-access-zt74d" pod "e6124ae8-fca1-4a31-ba83-145b7027f136" (UID: "e6124ae8-fca1-4a31-ba83-145b7027f136"). InnerVolumeSpecName "kube-api-access-zt74d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:47:45.640069 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:45.639967    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-xks-ca" pod "e6124ae8-fca1-4a31-ba83-145b7027f136" (UID: "e6124ae8-fca1-4a31-ba83-145b7027f136"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:47:45.739560 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:45.739479    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:47:45.739560 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:45.739518    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt74d\" (UniqueName: \"kubernetes.io/projected/e6124ae8-fca1-4a31-ba83-145b7027f136-kube-api-access-zt74d\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:47:45.739560 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:45.739535    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6124ae8-fca1-4a31-ba83-145b7027f136-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:47:46.415206 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:46.415178    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb"
Oct 29 23:47:46.415356 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:46.415212    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-f84xb" event={"ID":"e6124ae8-fca1-4a31-ba83-145b7027f136","Type":"ContainerDied","Data":"badfa6ced1145a870e763d0956ff86edf7e8fb854f2ec48890d749a08cdb2da4"}
Oct 29 23:47:46.415356 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:46.415282    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="badfa6ced1145a870e763d0956ff86edf7e8fb854f2ec48890d749a08cdb2da4"
Oct 29 23:47:49.809559 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:49.809524    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:47:55.506200 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.506165    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"]
Oct 29 23:47:55.506689 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.506534    6040 cpu_manager.go:414] "containerMap: removing container" podUID="e6124ae8-fca1-4a31-ba83-145b7027f136" containerName="revoke-keys"
Oct 29 23:47:55.506689 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.506545    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6124ae8-fca1-4a31-ba83-145b7027f136" containerName="revoke-keys"
Oct 29 23:47:55.506689 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.506608    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6124ae8-fca1-4a31-ba83-145b7027f136" containerName="revoke-keys"
Oct 29 23:47:55.509960 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.509936    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:55.530703 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.530674    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"]
Oct 29 23:47:55.539404 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.539383    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:47:55.539914 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.539899    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:47:55.539914 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.539898    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:47:55.540064 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.539931    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:47:55.677629 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.677596    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-x95cb\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:55.677910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.677676    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nfw2\" (UniqueName: \"kubernetes.io/projected/1e7a3704-3450-4311-b0d0-d63b5296b960-kube-api-access-5nfw2\") pod \"maas-api-revoke-sub-0165d7d2-x95cb\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:55.677910 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.677788    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-x95cb\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:55.778416 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.778340    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nfw2\" (UniqueName: \"kubernetes.io/projected/1e7a3704-3450-4311-b0d0-d63b5296b960-kube-api-access-5nfw2\") pod \"maas-api-revoke-sub-0165d7d2-x95cb\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:55.778416 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.778388    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-x95cb\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:55.778598 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.778434    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-x95cb\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:55.779083 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.779058    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-x95cb\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:55.780694 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.780673    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-x95cb\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:55.801103 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.801079    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nfw2\" (UniqueName: \"kubernetes.io/projected/1e7a3704-3450-4311-b0d0-d63b5296b960-kube-api-access-5nfw2\") pod \"maas-api-revoke-sub-0165d7d2-x95cb\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:55.822936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:55.822915    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:56.119348 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:56.119320    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"]
Oct 29 23:47:56.121365 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:47:56.121336    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e7a3704_3450_4311_b0d0_d63b5296b960.slice/crio-1a0e18a20147c9848df71ff4f99c570f4881115ff342a431c58155688ecfaea7 WatchSource:0}: Error finding container 1a0e18a20147c9848df71ff4f99c570f4881115ff342a431c58155688ecfaea7: Status 404 returned error can't find the container with id 1a0e18a20147c9848df71ff4f99c570f4881115ff342a431c58155688ecfaea7
Oct 29 23:47:56.485630 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:56.485598    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb" event={"ID":"1e7a3704-3450-4311-b0d0-d63b5296b960","Type":"ContainerStarted","Data":"a257d0f416319dcfe2042c0569966fe4bc617b47bcf1fe5d8eed675a5b9c7709"}
Oct 29 23:47:56.485804 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:56.485646    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb" event={"ID":"1e7a3704-3450-4311-b0d0-d63b5296b960","Type":"ContainerStarted","Data":"1a0e18a20147c9848df71ff4f99c570f4881115ff342a431c58155688ecfaea7"}
Oct 29 23:47:56.526218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:56.526166    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb" podStartSLOduration=1.52615236 podStartE2EDuration="1.52615236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:47:56.525188559 +0000 UTC m=+2056.220297437" watchObservedRunningTime="2026-10-29 23:47:56.52615236 +0000 UTC m=+2056.221261229"
Oct 29 23:47:57.491479 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:57.491446    6040 generic.go:356] "Generic (PLEG): container finished" podID="1e7a3704-3450-4311-b0d0-d63b5296b960" containerID="a257d0f416319dcfe2042c0569966fe4bc617b47bcf1fe5d8eed675a5b9c7709" exitCode=0
Oct 29 23:47:57.491651 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:57.491530    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb" event={"ID":"1e7a3704-3450-4311-b0d0-d63b5296b960","Type":"ContainerDied","Data":"a257d0f416319dcfe2042c0569966fe4bc617b47bcf1fe5d8eed675a5b9c7709"}
Oct 29 23:47:58.632994 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:58.632972    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:58.807696 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:58.807607    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e7a3704-3450-4311-b0d0-d63b5296b960-kube-api-access-5nfw2\" (UniqueName: \"kubernetes.io/projected/1e7a3704-3450-4311-b0d0-d63b5296b960-kube-api-access-5nfw2\") pod \"1e7a3704-3450-4311-b0d0-d63b5296b960\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") "
Oct 29 23:47:58.807696 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:58.807652    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-xks-ca\") pod \"1e7a3704-3450-4311-b0d0-d63b5296b960\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") "
Oct 29 23:47:58.807963 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:58.807763    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-openshift-ca\") pod \"1e7a3704-3450-4311-b0d0-d63b5296b960\" (UID: \"1e7a3704-3450-4311-b0d0-d63b5296b960\") "
Oct 29 23:47:58.808163 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:58.808137    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-openshift-ca" pod "1e7a3704-3450-4311-b0d0-d63b5296b960" (UID: "1e7a3704-3450-4311-b0d0-d63b5296b960"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:47:58.809798 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:58.809775    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-xks-ca" pod "1e7a3704-3450-4311-b0d0-d63b5296b960" (UID: "1e7a3704-3450-4311-b0d0-d63b5296b960"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:47:58.809886 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:58.809779    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e7a3704-3450-4311-b0d0-d63b5296b960-kube-api-access-5nfw2" pod "1e7a3704-3450-4311-b0d0-d63b5296b960" (UID: "1e7a3704-3450-4311-b0d0-d63b5296b960"). InnerVolumeSpecName "kube-api-access-5nfw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:47:58.909072 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:58.909038    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:47:58.909072 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:58.909066    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nfw2\" (UniqueName: \"kubernetes.io/projected/1e7a3704-3450-4311-b0d0-d63b5296b960-kube-api-access-5nfw2\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:47:58.909072 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:58.909079    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e7a3704-3450-4311-b0d0-d63b5296b960-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:47:59.505351 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:59.505321    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb"
Oct 29 23:47:59.505351 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:59.505328    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x95cb" event={"ID":"1e7a3704-3450-4311-b0d0-d63b5296b960","Type":"ContainerDied","Data":"1a0e18a20147c9848df71ff4f99c570f4881115ff342a431c58155688ecfaea7"}
Oct 29 23:47:59.505556 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:47:59.505372    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a0e18a20147c9848df71ff4f99c570f4881115ff342a431c58155688ecfaea7"
Oct 29 23:47:59.778034 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:47:59.777949    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:48:06.512682 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.512647    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"]
Oct 29 23:48:06.513156 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.513053    6040 cpu_manager.go:414] "containerMap: removing container" podUID="1e7a3704-3450-4311-b0d0-d63b5296b960" containerName="revoke-keys"
Oct 29 23:48:06.513156 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.513064    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e7a3704-3450-4311-b0d0-d63b5296b960" containerName="revoke-keys"
Oct 29 23:48:06.513156 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.513147    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e7a3704-3450-4311-b0d0-d63b5296b960" containerName="revoke-keys"
Oct 29 23:48:06.516089 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.516068    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:06.536633 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.536588    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"]
Oct 29 23:48:06.561339 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.561304    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:48:06.561517 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.561370    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fp9sw\""
Oct 29 23:48:06.569060 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.569006    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:48:06.569194 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.569087    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:48:06.573523 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.573493    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-rkplw\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:06.573619 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.573539    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-rkplw\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:06.573688 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.573668    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj7t4\" (UniqueName: \"kubernetes.io/projected/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-kube-api-access-qj7t4\") pod \"maas-api-revoke-sub-6b19338f-rkplw\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:06.674936 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.674895    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj7t4\" (UniqueName: \"kubernetes.io/projected/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-kube-api-access-qj7t4\") pod \"maas-api-revoke-sub-6b19338f-rkplw\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:06.675140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.674970    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-rkplw\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:06.675140 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.675012    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-rkplw\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:06.675671 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.675642    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-rkplw\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:06.677360 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.677337    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-rkplw\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:06.698004 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.697976    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj7t4\" (UniqueName: \"kubernetes.io/projected/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-kube-api-access-qj7t4\") pod \"maas-api-revoke-sub-6b19338f-rkplw\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:06.828711 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:06.828625    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:07.124427 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:07.124321    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"]
Oct 29 23:48:07.126642 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:48:07.126615    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bb5e1c7_2f84_4870_a4fe_e9f5c098edb4.slice/crio-97fbdfc34aa3f5c7345b534f979abc5a74637c7ac56f636739f18bcca057f445 WatchSource:0}: Error finding container 97fbdfc34aa3f5c7345b534f979abc5a74637c7ac56f636739f18bcca057f445: Status 404 returned error can't find the container with id 97fbdfc34aa3f5c7345b534f979abc5a74637c7ac56f636739f18bcca057f445
Oct 29 23:48:07.549134 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:07.549100    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw" event={"ID":"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4","Type":"ContainerStarted","Data":"8d439823c404c46dd4fdfc590ef08a885e9c7688de019a635e3e6cfa4d44a9b1"}
Oct 29 23:48:07.549528 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:07.549161    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw" event={"ID":"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4","Type":"ContainerStarted","Data":"97fbdfc34aa3f5c7345b534f979abc5a74637c7ac56f636739f18bcca057f445"}
Oct 29 23:48:07.589182 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:07.589104    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw" podStartSLOduration=1.589087208 podStartE2EDuration="1.589087208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:48:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:48:07.588023325 +0000 UTC m=+2067.283132207" watchObservedRunningTime="2026-10-29 23:48:07.589087208 +0000 UTC m=+2067.284196087"
Oct 29 23:48:08.555099 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:08.555065    6040 generic.go:356] "Generic (PLEG): container finished" podID="5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4" containerID="8d439823c404c46dd4fdfc590ef08a885e9c7688de019a635e3e6cfa4d44a9b1" exitCode=0
Oct 29 23:48:08.555477 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:08.555149    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw" event={"ID":"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4","Type":"ContainerDied","Data":"8d439823c404c46dd4fdfc590ef08a885e9c7688de019a635e3e6cfa4d44a9b1"}
Oct 29 23:48:09.693701 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:09.693676    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:09.800665 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:09.800627    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-kube-api-access-qj7t4\" (UniqueName: \"kubernetes.io/projected/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-kube-api-access-qj7t4\") pod \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") "
Oct 29 23:48:09.800860 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:09.800694    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-xks-ca\") pod \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") "
Oct 29 23:48:09.800860 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:09.800801    6040 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-openshift-ca\") pod \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\" (UID: \"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4\") "
Oct 29 23:48:09.801139 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:09.801109    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-openshift-ca" pod "5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4" (UID: "5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:48:09.802653 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:09.802634    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-xks-ca" pod "5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4" (UID: "5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:48:09.802729 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:09.802709    6040 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-kube-api-access-qj7t4" pod "5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4" (UID: "5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4"). InnerVolumeSpecName "kube-api-access-qj7t4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:48:09.902057 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:09.902025    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-openshift-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:48:09.902057 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:09.902053    6040 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj7t4\" (UniqueName: \"kubernetes.io/projected/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-kube-api-access-qj7t4\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:48:09.902057 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:09.902064    6040 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4-maas-api-xks-ca\") on node \"ip-10-0-138-189.ec2.internal\" DevicePath \"\""
Oct 29 23:48:10.566160 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:10.566133    6040 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw"
Oct 29 23:48:10.566328 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:10.566168    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-rkplw" event={"ID":"5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4","Type":"ContainerDied","Data":"97fbdfc34aa3f5c7345b534f979abc5a74637c7ac56f636739f18bcca057f445"}
Oct 29 23:48:10.566328 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:10.566207    6040 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97fbdfc34aa3f5c7345b534f979abc5a74637c7ac56f636739f18bcca057f445"
Oct 29 23:48:17.690655 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:17.690621    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2mz5s"]
Oct 29 23:48:28.981190 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:28.981165    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-qd5tw_5f6782eb-47b3-4936-ace2-0836f4821633/manager/0.log"
Oct 29 23:48:29.171625 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:29.171594    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-cdpft_d3a9d783-e528-4069-aeab-358b1af624cb/manager/0.log"
Oct 29 23:48:29.380122 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:29.380044    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-jhwdt_e39453e6-07f8-40ba-9622-64e5e1c70fe7/manager/0.log"
Oct 29 23:48:29.779393 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:48:29.779355    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:48:29.884064 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:29.884039    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-5vwwh_0483ae56-3819-4943-9d51-a18dd82502bc/server/0.log"
Oct 29 23:48:31.029622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:31.029593    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57666cc899-t5nwr_175dfb73-992b-404b-a72d-562b052cba0a/copy-manifests/0.log"
Oct 29 23:48:31.064023 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:31.063993    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57666cc899-t5nwr_175dfb73-992b-404b-a72d-562b052cba0a/manager/0.log"
Oct 29 23:48:32.458450 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:32.458424    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-x95cb_1e7a3704-3450-4311-b0d0-d63b5296b960/revoke-keys/0.log"
Oct 29 23:48:33.217689 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:33.217658    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-rkplw_5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4/revoke-keys/0.log"
Oct 29 23:48:33.945647 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:33.945621    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-f84xb_e6124ae8-fca1-4a31-ba83-145b7027f136/revoke-keys/0.log"
Oct 29 23:48:36.139330 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:36.139297    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-jx6jl_6acc30b0-f8c4-4b4c-a625-9290959971e3/manager/0.log"
Oct 29 23:48:36.316368 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:36.316341    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-t522s_95520c7b-e7d6-4257-a9a5-1f102d87fe26/kuadrant-console-plugin/0.log"
Oct 29 23:48:36.746413 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:36.746383    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-2mz5s_861c82d8-daf1-46e1-bc2f-a4a3e6b72262/limitador/0.log"
Oct 29 23:48:37.758239 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:37.758211    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-557c4f746fjz9lk_ed971fe7-998b-462e-bd88-3bb7001aa163/istio-proxy/0.log"
Oct 29 23:48:39.135157 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:39.135128    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-dh2pt_07f389b2-8dd8-4149-ad51-bacbf9f1adbf/istio-proxy/1.log"
Oct 29 23:48:39.326591 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:39.326556    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-lbpk9_3eaa91d2-99cc-4e70-b98f-0c5a851dfb34/istio-proxy/0.log"
Oct 29 23:48:39.934970 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:39.934924    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-pb266_41115057-f034-42e6-96c0-1b398529a14c/istio-proxy/0.log"
Oct 29 23:48:41.365218 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:41.365190    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-dh2pt_07f389b2-8dd8-4149-ad51-bacbf9f1adbf/istio-proxy/0.log"
Oct 29 23:48:53.569320 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:53.569289    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-qd5tw_5f6782eb-47b3-4936-ace2-0836f4821633/manager/0.log"
Oct 29 23:48:53.775462 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:53.775435    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-cdpft_d3a9d783-e528-4069-aeab-358b1af624cb/manager/0.log"
Oct 29 23:48:54.016982 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:54.016949    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-jhwdt_e39453e6-07f8-40ba-9622-64e5e1c70fe7/manager/0.log"
Oct 29 23:48:54.461402 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:54.461372    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-5vwwh_0483ae56-3819-4943-9d51-a18dd82502bc/server/0.log"
Oct 29 23:48:55.607973 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:55.607937    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57666cc899-t5nwr_175dfb73-992b-404b-a72d-562b052cba0a/manager/0.log"
Oct 29 23:48:55.640543 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:55.640515    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57666cc899-t5nwr_175dfb73-992b-404b-a72d-562b052cba0a/copy-manifests/0.log"
Oct 29 23:48:56.749678 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:56.749647    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-x95cb_1e7a3704-3450-4311-b0d0-d63b5296b960/revoke-keys/0.log"
Oct 29 23:48:57.463216 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:57.463191    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-rkplw_5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4/revoke-keys/0.log"
Oct 29 23:48:58.225124 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:48:58.225095    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-f84xb_e6124ae8-fca1-4a31-ba83-145b7027f136/revoke-keys/0.log"
Oct 29 23:48:59.777888 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:48:59.777855    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:49:00.036380 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:49:00.036301    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-jx6jl_6acc30b0-f8c4-4b4c-a625-9290959971e3/manager/0.log"
Oct 29 23:49:00.214313 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:49:00.214277    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-t522s_95520c7b-e7d6-4257-a9a5-1f102d87fe26/kuadrant-console-plugin/0.log"
Oct 29 23:49:00.624010 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:49:00.623981    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-2mz5s_861c82d8-daf1-46e1-bc2f-a4a3e6b72262/limitador/0.log"
Oct 29 23:49:01.413153 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:49:01.413126    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-557c4f746fjz9lk_ed971fe7-998b-462e-bd88-3bb7001aa163/istio-proxy/0.log"
Oct 29 23:49:02.756990 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:49:02.756958    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-dh2pt_07f389b2-8dd8-4149-ad51-bacbf9f1adbf/istio-proxy/1.log"
Oct 29 23:49:02.951223 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:49:02.951194    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-lbpk9_3eaa91d2-99cc-4e70-b98f-0c5a851dfb34/istio-proxy/0.log"
Oct 29 23:49:03.501883 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:49:03.501857    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-pb266_41115057-f034-42e6-96c0-1b398529a14c/istio-proxy/0.log"
Oct 29 23:49:29.779143 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:49:29.779103    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:49:59.778995 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:49:59.778896    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:50:26.781658 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.781617    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w8hzv/must-gather-p9fh2"]
Oct 29 23:50:26.784094 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.782034    6040 cpu_manager.go:414] "containerMap: removing container" podUID="5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4" containerName="revoke-keys"
Oct 29 23:50:26.784094 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.782046    6040 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4" containerName="revoke-keys"
Oct 29 23:50:26.784094 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.782129    6040 memory_manager.go:365] "RemoveStaleState removing state" podUID="5bb5e1c7-2f84-4870-a4fe-e9f5c098edb4" containerName="revoke-keys"
Oct 29 23:50:26.785193 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.785178    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w8hzv/must-gather-p9fh2"
Oct 29 23:50:26.800289 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.800265    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w8hzv\"/\"openshift-service-ca.crt\""
Oct 29 23:50:26.800397 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.800330    6040 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w8hzv\"/\"kube-root-ca.crt\""
Oct 29 23:50:26.812228 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.812208    6040 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w8hzv\"/\"default-dockercfg-l6v7j\""
Oct 29 23:50:26.823618 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.823601    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe-must-gather-output\") pod \"must-gather-p9fh2\" (UID: \"a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe\") " pod="openshift-must-gather-w8hzv/must-gather-p9fh2"
Oct 29 23:50:26.823679 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.823633    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h79wh\" (UniqueName: \"kubernetes.io/projected/a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe-kube-api-access-h79wh\") pod \"must-gather-p9fh2\" (UID: \"a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe\") " pod="openshift-must-gather-w8hzv/must-gather-p9fh2"
Oct 29 23:50:26.871192 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.871169    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w8hzv/must-gather-p9fh2"]
Oct 29 23:50:26.924397 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.924368    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe-must-gather-output\") pod \"must-gather-p9fh2\" (UID: \"a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe\") " pod="openshift-must-gather-w8hzv/must-gather-p9fh2"
Oct 29 23:50:26.924548 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.924405    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h79wh\" (UniqueName: \"kubernetes.io/projected/a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe-kube-api-access-h79wh\") pod \"must-gather-p9fh2\" (UID: \"a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe\") " pod="openshift-must-gather-w8hzv/must-gather-p9fh2"
Oct 29 23:50:26.924696 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.924679    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe-must-gather-output\") pod \"must-gather-p9fh2\" (UID: \"a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe\") " pod="openshift-must-gather-w8hzv/must-gather-p9fh2"
Oct 29 23:50:26.951384 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:26.951361    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h79wh\" (UniqueName: \"kubernetes.io/projected/a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe-kube-api-access-h79wh\") pod \"must-gather-p9fh2\" (UID: \"a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe\") " pod="openshift-must-gather-w8hzv/must-gather-p9fh2"
Oct 29 23:50:27.098932 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:27.098843    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w8hzv/must-gather-p9fh2"
Oct 29 23:50:27.393580 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:27.393556    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w8hzv/must-gather-p9fh2"]
Oct 29 23:50:27.395156 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:50:27.395120    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9cbbe8a_03f9_4aa8_b783_c72b6c8223fe.slice/crio-d7a7de37e0a06566e55a0f0c077c13cbd6fcb31427dd9ccc4b01dc0d510e96c3 WatchSource:0}: Error finding container d7a7de37e0a06566e55a0f0c077c13cbd6fcb31427dd9ccc4b01dc0d510e96c3: Status 404 returned error can't find the container with id d7a7de37e0a06566e55a0f0c077c13cbd6fcb31427dd9ccc4b01dc0d510e96c3
Oct 29 23:50:28.274537 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:28.274504    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8hzv/must-gather-p9fh2" event={"ID":"a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe","Type":"ContainerStarted","Data":"d7a7de37e0a06566e55a0f0c077c13cbd6fcb31427dd9ccc4b01dc0d510e96c3"}
Oct 29 23:50:29.287241 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:29.287196    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8hzv/must-gather-p9fh2" event={"ID":"a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe","Type":"ContainerStarted","Data":"9f92284c9c89a64ab9d67354a25e0c094e8ab8c4dd0409a0c8d275f157d03b04"}
Oct 29 23:50:29.287725 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:29.287259    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8hzv/must-gather-p9fh2" event={"ID":"a9cbbe8a-03f9-4aa8-b783-c72b6c8223fe","Type":"ContainerStarted","Data":"14f05254eb9b442ce3a0e3e195ba038a2159e81043a5b18c1d680ecbbc5a26ee"}
Oct 29 23:50:29.329264 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:29.329216    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-w8hzv/must-gather-p9fh2" podStartSLOduration=2.557707764 podStartE2EDuration="3.329200789s" totalImagesPullingTime="771.493025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:50:29.326615095 +0000 UTC m=+2209.021723955" watchObservedRunningTime="2026-10-29 23:50:29.329200789 +0000 UTC m=+2209.024309666"
Oct 29 23:50:29.783507 ip-10-0-138-189 kubenswrapper[6040]: E1029 23:50:29.783470    6040 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fk7ml" containerName="extractor"
Oct 29 23:50:31.278054 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:31.278027    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wht7s_a8ac2ba2-0d4b-4498-a7c6-658412d60ccf/global-pull-secret-syncer/0.log"
Oct 29 23:50:32.132761 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:32.128145    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5z4zk_9271cfd4-96c6-46a0-84c9-dcdcdccaae4e/konnectivity-agent/0.log"
Oct 29 23:50:32.417044 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:32.416959    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-189.ec2.internal_2f6cf61a946e2452e67c50b8a0479c0d/haproxy/0.log"
Oct 29 23:50:38.963353 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:38.963326    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-jx6jl_6acc30b0-f8c4-4b4c-a625-9290959971e3/manager/0.log"
Oct 29 23:50:39.098762 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:39.098704    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-t522s_95520c7b-e7d6-4257-a9a5-1f102d87fe26/kuadrant-console-plugin/0.log"
Oct 29 23:50:39.369545 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:39.369453    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-2mz5s_861c82d8-daf1-46e1-bc2f-a4a3e6b72262/limitador/0.log"
Oct 29 23:50:40.754781 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.754720    6040 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"]
Oct 29 23:50:40.760891 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.760867    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.781650 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.781624    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"]
Oct 29 23:50:40.879500 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.879462    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-podres\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.879695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.879520    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-sys\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.879695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.879618    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-lib-modules\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.879695 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.879660    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-proc\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.879882 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.879724    6040 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l4x7\" (UniqueName: \"kubernetes.io/projected/f16e78b4-5d18-4909-96a0-3af6d5a4b637-kube-api-access-2l4x7\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.980438 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.980395    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-podres\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.980622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.980457    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-sys\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.980622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.980497    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-lib-modules\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.980622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.980530    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-proc\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.980622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.980556    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-podres\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.980622 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.980564    6040 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l4x7\" (UniqueName: \"kubernetes.io/projected/f16e78b4-5d18-4909-96a0-3af6d5a4b637-kube-api-access-2l4x7\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.980880 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.980623    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-sys\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.980880 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.980702    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-lib-modules\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.980880 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.980707    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f16e78b4-5d18-4909-96a0-3af6d5a4b637-proc\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:40.981549 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:40.981530    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1628ae69-8a7c-4ef2-a409-add5a2a379c0/alertmanager/0.log"
Oct 29 23:50:41.004922 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:41.004076    6040 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l4x7\" (UniqueName: \"kubernetes.io/projected/f16e78b4-5d18-4909-96a0-3af6d5a4b637-kube-api-access-2l4x7\") pod \"perf-node-gather-daemonset-dhftd\" (UID: \"f16e78b4-5d18-4909-96a0-3af6d5a4b637\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:41.085012 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:41.084978    6040 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:41.118592 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:41.118568    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1628ae69-8a7c-4ef2-a409-add5a2a379c0/config-reloader/0.log"
Oct 29 23:50:41.249881 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:41.249853    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1628ae69-8a7c-4ef2-a409-add5a2a379c0/kube-rbac-proxy-web/0.log"
Oct 29 23:50:41.377918 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:41.377837    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1628ae69-8a7c-4ef2-a409-add5a2a379c0/kube-rbac-proxy/0.log"
Oct 29 23:50:41.394697 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:41.394673    6040 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"]
Oct 29 23:50:41.397352 ip-10-0-138-189 kubenswrapper[6040]: W1029 23:50:41.397325    6040 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf16e78b4_5d18_4909_96a0_3af6d5a4b637.slice/crio-202d32284d60d3601397407e4738976222c084c732bda0d6a84bf1a9438ffd0b WatchSource:0}: Error finding container 202d32284d60d3601397407e4738976222c084c732bda0d6a84bf1a9438ffd0b: Status 404 returned error can't find the container with id 202d32284d60d3601397407e4738976222c084c732bda0d6a84bf1a9438ffd0b
Oct 29 23:50:41.508017 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:41.507988    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1628ae69-8a7c-4ef2-a409-add5a2a379c0/kube-rbac-proxy-metric/0.log"
Oct 29 23:50:41.636879 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:41.636854    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1628ae69-8a7c-4ef2-a409-add5a2a379c0/prom-label-proxy/0.log"
Oct 29 23:50:41.766471 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:41.766440    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1628ae69-8a7c-4ef2-a409-add5a2a379c0/init-config-reloader/0.log"
Oct 29 23:50:42.400143 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:42.400110    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd" event={"ID":"f16e78b4-5d18-4909-96a0-3af6d5a4b637","Type":"ContainerStarted","Data":"106f3ecf016a6ac3440cade2f998437b921b8849ff4c9fbde27fdd778b375888"}
Oct 29 23:50:42.400337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:42.400162    6040 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd" event={"ID":"f16e78b4-5d18-4909-96a0-3af6d5a4b637","Type":"ContainerStarted","Data":"202d32284d60d3601397407e4738976222c084c732bda0d6a84bf1a9438ffd0b"}
Oct 29 23:50:42.400337 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:42.400226    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:42.422248 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:42.422225    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-78f9fd9c95-fj6xw_cbf0e826-b477-47a2-9c6d-08ac105aedd0/metrics-server/0.log"
Oct 29 23:50:42.444843 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:42.444795    6040 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd" podStartSLOduration=2.444778629 podStartE2EDuration="2.444778629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:50:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:50:42.442542461 +0000 UTC m=+2222.137651339" watchObservedRunningTime="2026-10-29 23:50:42.444778629 +0000 UTC m=+2222.139887507"
Oct 29 23:50:42.685873 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:42.685804    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dn5kk_59d798e8-802a-4a6a-8f5a-063d69972c13/node-exporter/0.log"
Oct 29 23:50:42.812028 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:42.811998    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dn5kk_59d798e8-802a-4a6a-8f5a-063d69972c13/kube-rbac-proxy/0.log"
Oct 29 23:50:42.940214 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:42.940133    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dn5kk_59d798e8-802a-4a6a-8f5a-063d69972c13/init-textfile/0.log"
Oct 29 23:50:45.597892 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:45.597861    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-79699f4d76-w6hb5_259771b7-e14a-496a-bba1-a8f7364e6c1f/telemeter-client/0.log"
Oct 29 23:50:45.725355 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:45.725331    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-79699f4d76-w6hb5_259771b7-e14a-496a-bba1-a8f7364e6c1f/reload/0.log"
Oct 29 23:50:45.850993 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:45.850904    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-79699f4d76-w6hb5_259771b7-e14a-496a-bba1-a8f7364e6c1f/kube-rbac-proxy/0.log"
Oct 29 23:50:48.417425 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:48.417395    6040 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-dhftd"
Oct 29 23:50:50.467516 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:50.467463    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6d74c4fd6f-gwb47_6df68024-d974-4b31-9858-092da3b9eb9d/console/0.log"
Oct 29 23:50:50.607015 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:50.606985    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-9jhw5_ed669269-4708-416f-8b97-9ee2cc5bf891/download-server/0.log"
Oct 29 23:50:53.407217 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:53.407176    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-557c4f746fjz9lk_ed971fe7-998b-462e-bd88-3bb7001aa163/istio-proxy/0.log"
Oct 29 23:50:54.663582 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:54.663555    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mnl8f_802fe00a-6d8f-483e-9fb5-2ea266688ad1/dns/0.log"
Oct 29 23:50:54.789845 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:54.789809    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mnl8f_802fe00a-6d8f-483e-9fb5-2ea266688ad1/kube-rbac-proxy/0.log"
Oct 29 23:50:54.881709 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:54.881683    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-dh2pt_07f389b2-8dd8-4149-ad51-bacbf9f1adbf/istio-proxy/0.log"
Oct 29 23:50:54.882993 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:54.882939    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-dh2pt_07f389b2-8dd8-4149-ad51-bacbf9f1adbf/istio-proxy/1.log"
Oct 29 23:50:54.919504 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:54.919436    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n4lkv_eeeba32d-22c4-4173-83f0-5b4c391644b0/dns-node-resolver/0.log"
Oct 29 23:50:54.921547 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:54.921527    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-lbpk9_3eaa91d2-99cc-4e70-b98f-0c5a851dfb34/istio-proxy/0.log"
Oct 29 23:50:55.138699 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:55.138670    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-pb266_41115057-f034-42e6-96c0-1b398529a14c/istio-proxy/0.log"
Oct 29 23:50:56.754454 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:56.754426    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tn9mk_916781d5-af66-4d17-ba3f-748bab236eb1/node-ca/0.log"
Oct 29 23:50:57.795681 ip-10-0-138-189 kubenswrapper[6040]: I1029 23:50:57.795599    6040 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-557c4f746fjz9lk_ed971fe7-998b-462e-bd88-3bb7001aa163/istio-proxy/0.log"